-- Logs begin at Sat 2022-07-30 04:09:50 UTC, end at Sat 2022-07-30 05:16:05 UTC. -- Jul 30 04:43:55.990404 dv-cinder-851279-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 30 04:43:56.001399 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jul 30 04:43:56 2022] *** Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: nodename: dv-cinder-851279-3-iscsi Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: machine: x86_64 Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: clock source: unix Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: pcre jit disabled Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: detected number of CPU cores: 4 Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: current working directory: / Jul 30 04:43:56.003502 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: detected binary path: /usr/bin/uwsgi-core Jul 30 04:43:56.004025 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: your processes number limit is 47680 Jul 30 04:43:56.004025 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: your memory page size is 4096 bytes Jul 30 04:43:56.004025 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: detected max file descriptor number: 2048 Jul 30 04:43:56.004025 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: lock engine: pthread robust mutexes Jul 30 04:43:56.004025 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: thunder lock: enabled Jul 30 04:43:56.004025 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 30 04:43:56.004025 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 30 04:43:56.043211 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: Python main interpreter initialized at 0x55d0bda969c0 Jul 30 04:43:56.043211 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: python threads support enabled Jul 30 04:43:56.043211 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: your server socket listen backlog is limited to 100 connections Jul 30 04:43:56.043211 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: your mercy for graceful operations on workers is 90 seconds Jul 30 04:43:56.043562 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: mapped 403077 bytes (393 KB) for 2 cores Jul 30 04:43:56.043637 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: *** Operational MODE: preforking *** Jul 30 04:43:56.043918 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: *** uWSGI is running in multiple interpreter mode *** Jul 30 04:43:56.043918 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: spawned uWSGI master process (pid: 80158) Jul 30 04:43:56.044118 dv-cinder-851279-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 30 04:43:56.044674 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: spawned uWSGI worker 1 (pid: 80170, cores: 1) Jul 30 04:43:56.045088 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: spawned uWSGI worker 2 (pid: 80171, cores: 1) Jul 30 04:43:56.045245 dv-cinder-851279-3-iscsi devstack@keystone.service[80158]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 30 04:44:01.622838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 30 04:44:01.622838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: from cryptography.utils import int_from_bytes Jul 30 04:44:01.624164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 30 04:44:01.624164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: from cryptography.utils import int_from_bytes Jul 30 04:44:01.624945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 30 04:44:01.624945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 30 04:44:01.625406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 30 04:44:01.625406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 30 04:44:01.674830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 30 04:44:01.675025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 30 04:44:01.675218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] command line args: [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 30 04:44:01.675489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 30 04:44:01.675670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ================================================================================ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 30 04:44:01.675857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 30 04:44:01.675953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] admin_token = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.676052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 30 04:44:01.676143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config_dir = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.676248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] command line args: [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 30 04:44:01.676332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.676511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config_source = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.676593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 30 04:44:01.676692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] debug = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.676781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ================================================================================ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 30 04:44:01.676890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.677159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.677159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_dir = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.677266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.677363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.677435 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.677552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_source = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.677677 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.677757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] debug = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.677939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_config_append = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678715 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_dir = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.678983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.679136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.679302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.679461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_config_append = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.679597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.679773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.679908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.680027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_dir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.680027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.680235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.680349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.680459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.680459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.680677 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.680788 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.680902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.681012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.681131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.681257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.681369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.681369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.681583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.681700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.681818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.681938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.682057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.682175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.682292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.682402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.682553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.682683 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.682813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.682933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.682933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.683142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.683262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.683375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] publish_errors = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.683491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.683603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.683603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.683603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.683949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.684062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.684173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] publish_errors = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.684291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.684403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.684519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] standard_threads = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.684647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.684647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.684864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.684975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.685088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.685088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.685295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.685411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_journal = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.685587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] standard_threads = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.686304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_json = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.686448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.686565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_stderr = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.686565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_syslog = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.686789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.686903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.687020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.687020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_journal = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.687235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.687348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_json = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.687466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.687584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_stderr = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.687584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_syslog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.687792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.687908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 30 04:44:01.688025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.688145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.688269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.688388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.688505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.688617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.688733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.688856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.688974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.689085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.689196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.689313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.689425 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.689544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.689661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.689773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.689883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.689994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.690104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.690222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.690341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.690467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.690580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.690699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.690814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.690938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.691057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.691169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.691279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.691398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.691515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.691705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.691821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.691932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.692043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.692154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.692265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.692387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.692500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.692611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.692722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.692838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.692949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.693069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.693192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.693192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.693401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.693519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.693631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.693751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.693863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.693975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.694091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.694208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.694330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.694490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.694605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.694716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.694827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.694938 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.695060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.695179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.695291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.695402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.695513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.695624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.695745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.695857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.695968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.696078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.696188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.696298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.696420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.696532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.696649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.696761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.696877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.696989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.697173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.697287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.697398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.697509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.697620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.697735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.697853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.697965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.698076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.698193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.698305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.698415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.698553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.698666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.698779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.698896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.699946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.700039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.700131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.700224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.700350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.700457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.700561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.700672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.700784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.700903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.701015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.701134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.701247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.701411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.701555 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.701663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.701777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.701899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.702012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.702125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.702245 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.702355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.702508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.702623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.702623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.702849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.702963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.703076 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.703180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.703290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.703402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.703522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.703641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.703754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.703872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.703986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.704106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.704210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.704321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.704439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.704551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.704662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.704782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.704895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.705008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.705120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.705232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.705343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.705463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.705577 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.705690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.705802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.705920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.706032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.706153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.706266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.706377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.706503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.706618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.706731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.706854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.706969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.707081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.707200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.707319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.707432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.707552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.707670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.707782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.707901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.708014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.708125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.708313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.708428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.708428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.708646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.708758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.708870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.708995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.709116 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.709238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.709352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.709464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.709576 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.709688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.709800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.709920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.710034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.710034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.710255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.710364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.710500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.710628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.710742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.710861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.710973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.711086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.711198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.711318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.711432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.external = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.711543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.711655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.711767 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.711887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.712001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.712113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.712232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.external = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.712372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.password = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.712497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.712613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.token = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.712613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.712833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.712948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.713061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.713061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.password = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.713292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.713413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.token = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.713534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.713649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.713768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.713882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.713995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.714114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.714236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.714360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.714498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.714623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.714738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.714851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.714972 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.715087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.715201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.715314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.715427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.715540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.715664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.715664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.715877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.715877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.716089 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.716201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.716326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.716441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.716560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.716682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.716799 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.716922 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.717061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.717182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.717296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.717409 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.717527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.717642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.717763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.717877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.717998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.718113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.718226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.718339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.718502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.718619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.718733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.718846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.718959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.719071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.719196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.719312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.719431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.719545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.719658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.719772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.719892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.720013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.720127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.720240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.720353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.720467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.720587 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.720702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.720816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.720928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.721041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.721153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.721275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.721389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.721502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.721615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.721736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.721926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.722042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.722155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.722274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.722388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.722527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.722651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.722766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.722881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.722994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.723113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.723233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.723357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.723472 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.723585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.723698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.723811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.723925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.724045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.724160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.724279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.724393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.724578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.724695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.724816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.724933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.725047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.725161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.725275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.725388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.725509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.725630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.725744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.725858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.725972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.726086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.726207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.726322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.726451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.726567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.726681 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.726793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.726918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.727034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.727148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.727260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.727374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.727490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.727612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.727726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.727839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.727952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.728066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.728179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.728300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.728420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.728540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.728653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.728767 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.728880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.729001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.729116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.729228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.729341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.729454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.729567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.729694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.729809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.729922 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.730042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.730157 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.730276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.730402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.730574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.730689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.730802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.730916 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.731029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.731154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.731271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.731385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.731498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.731611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.731724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.731844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.731958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.732072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.732185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.732299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.732412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.733039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.733157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.733272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.733386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.733499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.733613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.733734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.733848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.733961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.734074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.734187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.734308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.734454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.734659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.734796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.734911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.735025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.735151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.735268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.735381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.735495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.735608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.735721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.735845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.735961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.736074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.736187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.736300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.736414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.736536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.736651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.736764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.736877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.736990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.737103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.737224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.737342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.737456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.737570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.737683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.737797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.737924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.738040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.738153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.738266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.738379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.738527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.738655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.738771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.738884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.738997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.739110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.739223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.739345 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.739460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.739573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.739687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.739800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.739913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.740035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.740150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.740263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.740376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.740489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.740602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.740723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.740838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.740951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.741064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.741177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.741290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.741410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.741525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.741639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.741753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.741871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.741986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.742107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.742222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.742414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.742546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.742660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.742775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.742900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.743016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.743129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.743242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.743355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.743469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.743590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.743705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.743820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.743933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.744046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.744159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.744280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.744395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.744508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.744621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.744734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.744849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.745039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.745156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.745270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.745383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.745496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.745609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.745731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.745846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.745960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.746137 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.746255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.746370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.746508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.746626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.746741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.746855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.746970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.747091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.747210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.747324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.747437 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.747550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.747663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.747785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.747899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.748015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.748128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.748836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.748954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.749078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.749193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.749307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.749427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.749542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.749655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.749777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.749901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.749901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.750122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.750235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.750348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.750528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.750646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.750760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.750873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.750986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.751099 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.751221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.751336 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.751449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.751563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.751676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.751789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.751911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.752026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.752139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.752253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.752366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.752486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.752601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.752714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.752827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.752939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.753053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.753173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.753289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.753402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.753515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.753627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.753740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.753861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.753975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.754089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.754201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.754315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.754427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.754572 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.754688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.754801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.754914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.755026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.755140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.755260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.755376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.755488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.755601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.755714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.755827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.755948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.756062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.756175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.756288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.756402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.756515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.756635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.756750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.756863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.756976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.757088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.757205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.757327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.757444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.757557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.757670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.757792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.757914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.758028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.758141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.758263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.758378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.758550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.758665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.758777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.758906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.759029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.759156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.759271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.759384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.759498 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.759612 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.759725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.759845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.759960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.760073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.760186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.760299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.760412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.760533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.760648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.760762 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.760875 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.760988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.761101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.761221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.761335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.761449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.761561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.761675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.761788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.761909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.761909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 30 04:44:01.762123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.762123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.762123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.762123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.762568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 30 04:44:01.762568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 30 04:44:01.790725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.790975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.791183 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.791517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.791753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.791979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.800258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.801082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.830928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.831155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.831241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.831479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.884756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.884952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.936655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.936888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.937026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:01.937108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.002312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.002508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.002642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.002747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.002850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.002950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.003062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.003165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.003277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.003376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.003474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.003581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.027407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.027520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.027652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.027792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.027926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.028015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.028153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.028236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.028372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.028466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.028586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.028668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.028801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.028882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.040572 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.040979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.054092 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.054214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.054326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.054412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.054576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.054661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.054799 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.054885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.055017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.055109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.055234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.055316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.055449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.055530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 30 04:44:02.072743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.072954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 30 04:44:02.073120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.073292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.073372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 30 04:44:02.073493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 30 04:44:02.073653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.073759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.073841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 30 04:44:02.074014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.074094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.074267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.074349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.074553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.074637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.074864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.084322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.084598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.084867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.084998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.085148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.085277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.085543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.085820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.092753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.093792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.097518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.097798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.098080 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.098359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.098584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.098668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.098866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.098960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.099150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.099232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.099417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.099498 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.099684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.099765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.099947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.100025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.100215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.100302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.100485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.100752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.101023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.101286 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.124439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.125408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.125932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.126213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.126935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.127220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.130924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.131201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.131478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.131883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.132163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.132445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.138249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.138519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.139192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.139477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.142857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.143145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.143435 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.143718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.143857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.143990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.144152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.144268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.144432 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.144542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.144712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.144799 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.144985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.145273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.145553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.145835 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.161694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.161966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.162765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.163049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.165973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.167018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.169164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.169457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.170243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.170625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.173993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.175096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.177225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.178297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.180454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.180753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.181050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.181334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.181504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.181623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.181804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.181896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.182091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.182380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.182681 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.182959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.198679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.198957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.199220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.199481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.199921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.200204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.200472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.200740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.207142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.208318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.208685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.208959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.209865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.210140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.211825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.212104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.212400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.212682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.212972 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.213055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.213264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.213567 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.213850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.214128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.224956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.225235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.225518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.225736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.225821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.226017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.226101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.226301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.226384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.226615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.226702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.226899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.227177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.227468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.242477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.243293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.245455 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.246278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.248994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.249801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.251493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.252310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.253148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.253432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.253713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.253860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.254143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.254424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.260288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.260988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.261904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.262623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.265449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.265736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.266022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.266276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.266359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.266592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.266879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.267167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.274769 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.275055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.275335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.275608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.275894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.276178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.282623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.282902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.283184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 30 04:44:02.283266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.283464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.283548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.283734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.283819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.284010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.284102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.284278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.284362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.284545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.284627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.284818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.284909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.285093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.285184 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.285459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.285586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.285722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.285804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.285992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.286073 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.286248 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.286339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.286540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.286804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 30 04:44:02.320873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d0bda969c0 pid: 80171 (default app) Jul 30 04:44:02.321443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d0bda969c0 pid: 80170 (default app) Jul 30 04:44:02.324200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174b6e0-f63c-4a4c-9fe7-40e07eb3790f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:02.324492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174b6e0-f63c-4a4c-9fe7-40e07eb3790f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:02.324769 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174b6e0-f63c-4a4c-9fe7-40e07eb3790f None None] PATH_INFO: `/v3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:02.329785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Sat Jul 30 04:44:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:09.951234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1436b-16df-4d0e-bb9b-053153086b91 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:09.952388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1436b-16df-4d0e-bb9b-053153086b91 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:09.952388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1436b-16df-4d0e-bb9b-053153086b91 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:09.956815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:44:09.961905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b7f95-5090-4663-8aae-744b87004935 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:09.962333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b7f95-5090-4663-8aae-744b87004935 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:09.962462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b7f95-5090-4663-8aae-744b87004935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:10.035004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: INFO dbcounter [None req-669b7f95-5090-4663-8aae-744b87004935 None None] Registered counter for database keystone Jul 30 04:44:10.054479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG oslo_db.sqlalchemy.engines [None req-669b7f95-5090-4663-8aae-744b87004935 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80171) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 30 04:44:10.060697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writer thread running {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 30 04:44:10.240549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG passlib.handlers.bcrypt [None req-669b7f95-5090-4663-8aae-744b87004935 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80171) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 30 04:44:10.240893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG passlib.handlers.bcrypt [None req-669b7f95-5090-4663-8aae-744b87004935 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80171) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 30 04:44:10.295411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-669b7f95-5090-4663-8aae-744b87004935 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:10.297144 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-669b7f95-5090-4663-8aae-744b87004935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:10.429476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 30 04:44:10.552943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1436b-16df-4d0e-bb9b-053153086b91 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:10.623493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: INFO dbcounter [None req-2df1436b-16df-4d0e-bb9b-053153086b91 None None] Registered counter for database keystone Jul 30 04:44:10.645406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG oslo_db.sqlalchemy.engines [None req-2df1436b-16df-4d0e-bb9b-053153086b91 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80170) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 30 04:44:10.651786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writer thread running {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 30 04:44:10.769583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1436b-16df-4d0e-bb9b-053153086b91 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:10.774172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28984adc-270c-4ebd-b1ee-e3fbe0ba9e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:10.775096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28984adc-270c-4ebd-b1ee-e3fbe0ba9e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:10.775377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28984adc-270c-4ebd-b1ee-e3fbe0ba9e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:10.775648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28984adc-270c-4ebd-b1ee-e3fbe0ba9e80 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:10.784578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28984adc-270c-4ebd-b1ee-e3fbe0ba9e80 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:10.837187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WARNING keystone.common.rbac_enforcer.enforcer [None req-28984adc-270c-4ebd-b1ee-e3fbe0ba9e80 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 30 04:44:10.837468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28984adc-270c-4ebd-b1ee-e3fbe0ba9e80 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:10.847942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Sat Jul 30 04:44:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 04:44:10.853923 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b7f95-5090-4663-8aae-744b87004935 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:10.869226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b7f95-5090-4663-8aae-744b87004935 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:10.874493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67f185-31ad-4842-a162-c7426b45404a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:10.875615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67f185-31ad-4842-a162-c7426b45404a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:10.875900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67f185-31ad-4842-a162-c7426b45404a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:10.876175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67f185-31ad-4842-a162-c7426b45404a admin admin] PATH_INFO: `/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:10.877668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67f185-31ad-4842-a162-c7426b45404a admin admin] RBAC: Authorizing `identity:get_project(project_id=2e75ff1f24d24ead87dfc7d32ddc1723)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:10.928256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4d67f185-31ad-4842-a162-c7426b45404a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 30 04:44:10.928608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67f185-31ad-4842-a162-c7426b45404a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:10.933209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Sat Jul 30 04:44:10 2022] GET /identity/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:12.906611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65d07d-c84f-47a9-bb7f-3f55cd75b2cf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:12.907162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65d07d-c84f-47a9-bb7f-3f55cd75b2cf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:12.907162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65d07d-c84f-47a9-bb7f-3f55cd75b2cf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:12.911271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:44:12.916658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e82d88-4acb-4cec-ab46-301636fe9e6e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:12.917084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e82d88-4acb-4cec-ab46-301636fe9e6e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:12.917266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e82d88-4acb-4cec-ab46-301636fe9e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:12.939066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a7e82d88-4acb-4cec-ab46-301636fe9e6e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:12.940599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a7e82d88-4acb-4cec-ab46-301636fe9e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:12.948682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 30 04:44:12.971143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65d07d-c84f-47a9-bb7f-3f55cd75b2cf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:12.983571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65d07d-c84f-47a9-bb7f-3f55cd75b2cf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:12.987825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012025dc-be29-44de-8591-fb038a8dcd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:12.988420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012025dc-be29-44de-8591-fb038a8dcd05 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:12.988698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012025dc-be29-44de-8591-fb038a8dcd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:12.988966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012025dc-be29-44de-8591-fb038a8dcd05 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:12.996619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012025dc-be29-44de-8591-fb038a8dcd05 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:12.998374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012025dc-be29-44de-8591-fb038a8dcd05 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:13.008984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Sat Jul 30 04:44:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 30 04:44:13.014462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e82d88-4acb-4cec-ab46-301636fe9e6e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:13.023551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e82d88-4acb-4cec-ab46-301636fe9e6e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:13.028232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a00eb-b782-479f-bc1d-a58d708c1af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:13.028869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a00eb-b782-479f-bc1d-a58d708c1af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:13.029146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a00eb-b782-479f-bc1d-a58d708c1af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:13.029418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a00eb-b782-479f-bc1d-a58d708c1af0 admin admin] PATH_INFO: `/v3/users/74c09f97e34c428a84e13d34f5105ca9` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:13.030918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a00eb-b782-479f-bc1d-a58d708c1af0 admin admin] RBAC: Authorizing `identity:get_user(user_id=74c09f97e34c428a84e13d34f5105ca9)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:13.032737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a00eb-b782-479f-bc1d-a58d708c1af0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:13.034301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Sat Jul 30 04:44:13 2022] GET /identity/v3/users/74c09f97e34c428a84e13d34f5105ca9 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:15.146363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4291cd2-0d7d-4629-bb8c-8c9df4d78431 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.146966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4291cd2-0d7d-4629-bb8c-8c9df4d78431 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.146966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4291cd2-0d7d-4629-bb8c-8c9df4d78431 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.148295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:15.153637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6b1bb-9711-4a87-9d83-d95895c58ccc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.154022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6b1bb-9711-4a87-9d83-d95895c58ccc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.154325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6b1bb-9711-4a87-9d83-d95895c58ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.174416 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-36e6b1bb-9711-4a87-9d83-d95895c58ccc None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:15.175640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-36e6b1bb-9711-4a87-9d83-d95895c58ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:15.253720 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 6/11] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:15.276277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4291cd2-0d7d-4629-bb8c-8c9df4d78431 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:15.323327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6051813-5dcf-46d5-9c32-11e461d3b3f8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.323618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6051813-5dcf-46d5-9c32-11e461d3b3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.323898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6051813-5dcf-46d5-9c32-11e461d3b3f8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.325268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 7/12] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:44:15.330671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2f183-d1c3-4b8b-859b-96a29f2002f2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.330959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2f183-d1c3-4b8b-859b-96a29f2002f2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.331234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2f183-d1c3-4b8b-859b-96a29f2002f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.349016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2fe2f183-d1c3-4b8b-859b-96a29f2002f2 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:15.350238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2fe2f183-d1c3-4b8b-859b-96a29f2002f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:15.367490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:15.423059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 8/13] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:15.427733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4291cd2-0d7d-4629-bb8c-8c9df4d78431 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:15.445725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2f183-d1c3-4b8b-859b-96a29f2002f2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:15.461575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff99cf-5d6d-43ba-984b-a701ae5106da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:15.462483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff99cf-5d6d-43ba-984b-a701ae5106da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.462750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff99cf-5d6d-43ba-984b-a701ae5106da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.463029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff99cf-5d6d-43ba-984b-a701ae5106da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.474701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff99cf-5d6d-43ba-984b-a701ae5106da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4aff99cf-5d6d-43ba-984b-a701ae5106da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:15.483953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:15.484878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:15.485693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:15.486976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 6/14] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:44:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:15.490733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff99cf-5d6d-43ba-984b-a701ae5106da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:15.525181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:15.548105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2f183-d1c3-4b8b-859b-96a29f2002f2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:15.582518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf6f06-7177-4677-96f2-08238714d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:15.583147 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf6f06-7177-4677-96f2-08238714d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.583423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf6f06-7177-4677-96f2-08238714d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.583693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf6f06-7177-4677-96f2-08238714d7b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.596297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff99cf-5d6d-43ba-984b-a701ae5106da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:15.608068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:15.633915 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42fd18-df85-4690-8af9-5dba2acdae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:15.634801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42fd18-df85-4690-8af9-5dba2acdae73 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.635230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42fd18-df85-4690-8af9-5dba2acdae73 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.635503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42fd18-df85-4690-8af9-5dba2acdae73 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.636221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42fd18-df85-4690-8af9-5dba2acdae73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:15.636524 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42fd18-df85-4690-8af9-5dba2acdae73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:15.637910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42fd18-df85-4690-8af9-5dba2acdae73 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:15.650280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 7/15] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:15.656377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42fd18-df85-4690-8af9-5dba2acdae73 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:15.658310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf6f06-7177-4677-96f2-08238714d7b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:15.659831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf6f06-7177-4677-96f2-08238714d7b3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:15.750954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:15.756905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42fd18-df85-4690-8af9-5dba2acdae73 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:15.758059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 9/16] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:44:15.763713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf6f06-7177-4677-96f2-08238714d7b3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:15.792223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce057c03-fb2c-4084-9e89-8b35c8d589bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:15.792862 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce057c03-fb2c-4084-9e89-8b35c8d589bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.793141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce057c03-fb2c-4084-9e89-8b35c8d589bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.793412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce057c03-fb2c-4084-9e89-8b35c8d589bc None admin] PATH_INFO: `/v3/users/74c09f97e34c428a84e13d34f5105ca9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.795031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce057c03-fb2c-4084-9e89-8b35c8d589bc None admin] RBAC: Authorizing `identity:get_user(user_id=74c09f97e34c428a84e13d34f5105ca9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:15.796572 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce057c03-fb2c-4084-9e89-8b35c8d589bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:15.798220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 8/17] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:15 2022] GET /identity/v3/users/74c09f97e34c428a84e13d34f5105ca9 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:15.803411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce057c03-fb2c-4084-9e89-8b35c8d589bc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:15.815904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:15.862329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf6f06-7177-4677-96f2-08238714d7b3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:15.897635 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e11c16-bfcf-43d2-9f7c-e55f7de6ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:15.898296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e11c16-bfcf-43d2-9f7c-e55f7de6ee23 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.898599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e11c16-bfcf-43d2-9f7c-e55f7de6ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.898879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e11c16-bfcf-43d2-9f7c-e55f7de6ee23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.905282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce057c03-fb2c-4084-9e89-8b35c8d589bc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:15.909409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e11c16-bfcf-43d2-9f7c-e55f7de6ee23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:15.910898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e11c16-bfcf-43d2-9f7c-e55f7de6ee23 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:15.940026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f659e-85c9-4894-a4e0-d8408aaed537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:15.940694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f659e-85c9-4894-a4e0-d8408aaed537 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:15.940981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f659e-85c9-4894-a4e0-d8408aaed537 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:15.941259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f659e-85c9-4894-a4e0-d8408aaed537 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:15.942643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f659e-85c9-4894-a4e0-d8408aaed537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:15.944131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f659e-85c9-4894-a4e0-d8408aaed537 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:15.945803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:15.952464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f659e-85c9-4894-a4e0-d8408aaed537 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:15.986095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e1e11c16-bfcf-43d2-9f7c-e55f7de6ee23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:44:16.037028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:16.037929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:16.038771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:44:16.038771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:44:16.038771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:44:16.038771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:16.038771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:44:16.044036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e11c16-bfcf-43d2-9f7c-e55f7de6ee23 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:16.061504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f659e-85c9-4894-a4e0-d8408aaed537 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:16.095797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3be4ae-497e-4921-ba41-4a0c59dcf793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:16.096411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3be4ae-497e-4921-ba41-4a0c59dcf793 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:16.096686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3be4ae-497e-4921-ba41-4a0c59dcf793 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:16.096962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3be4ae-497e-4921-ba41-4a0c59dcf793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:16.097743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3be4ae-497e-4921-ba41-4a0c59dcf793 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.domain.id=default, user.id=74c09f97e34c428a84e13d34f5105ca9) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:16.098044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3be4ae-497e-4921-ba41-4a0c59dcf793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:16.099508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3be4ae-497e-4921-ba41-4a0c59dcf793 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:16.107994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1322 bytes} [Sat Jul 30 04:44:15 2022] GET /identity/v3/role_assignments?user.id=74c09f97e34c428a84e13d34f5105ca9&scope.domain.id=default&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:16.153652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e11c16-bfcf-43d2-9f7c-e55f7de6ee23 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:16.157003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:16.192225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf12f10-c869-4034-a0b0-6b283fe2c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:16.192856 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf12f10-c869-4034-a0b0-6b283fe2c1ad None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:16.193138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf12f10-c869-4034-a0b0-6b283fe2c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:16.193412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf12f10-c869-4034-a0b0-6b283fe2c1ad None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:16.194117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf12f10-c869-4034-a0b0-6b283fe2c1ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:16.194419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf12f10-c869-4034-a0b0-6b283fe2c1ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:16.195865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf12f10-c869-4034-a0b0-6b283fe2c1ad None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:16.215349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:18.400727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f633803-2ade-4a67-b0af-2416480d5fcf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:18.401769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f633803-2ade-4a67-b0af-2416480d5fcf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:18.402251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f633803-2ade-4a67-b0af-2416480d5fcf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:18.403990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:18.409908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d036580-498e-418a-958c-4327a7a001aa None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:18.410729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d036580-498e-418a-958c-4327a7a001aa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:18.411223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d036580-498e-418a-958c-4327a7a001aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:18.417722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3a90a-d8e0-4706-a704-235e17a06cda None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:18.418012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3a90a-d8e0-4706-a704-235e17a06cda None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:18.418289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3a90a-d8e0-4706-a704-235e17a06cda None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:18.419926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 12/23] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:44:18.435487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3d036580-498e-418a-958c-4327a7a001aa None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:18.437036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3d036580-498e-418a-958c-4327a7a001aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:18.441967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc940ec-e57b-4821-bb24-46cbe1fb8328 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:18.442256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc940ec-e57b-4821-bb24-46cbe1fb8328 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:18.442563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc940ec-e57b-4821-bb24-46cbe1fb8328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:18.469883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-bbc940ec-e57b-4821-bb24-46cbe1fb8328 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80170) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 30 04:44:18.470317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-bbc940ec-e57b-4821-bb24-46cbe1fb8328 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80170) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 30 04:44:18.543583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bbc940ec-e57b-4821-bb24-46cbe1fb8328 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:18.545216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bbc940ec-e57b-4821-bb24-46cbe1fb8328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:18.564751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 12/24] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:18.607380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d036580-498e-418a-958c-4327a7a001aa None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:18.697980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:18.701259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719a5eb-d04a-48ac-b291-50361cfc3fec None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:18.701548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719a5eb-d04a-48ac-b291-50361cfc3fec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:18.701827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719a5eb-d04a-48ac-b291-50361cfc3fec None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:18.703916 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:18.709704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe8cb-d4b1-49a4-8eaa-6b7fea2619ce None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:18.710147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe8cb-d4b1-49a4-8eaa-6b7fea2619ce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:18.710427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe8cb-d4b1-49a4-8eaa-6b7fea2619ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:18.750083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e72fe8cb-d4b1-49a4-8eaa-6b7fea2619ce None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:18.751594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e72fe8cb-d4b1-49a4-8eaa-6b7fea2619ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:18.774841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:18.815356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:18.824185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d036580-498e-418a-958c-4327a7a001aa None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:18.873199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 15/27] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:18.874869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c83a35-6fb6-475c-8a71-e1c181c9e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:18.875674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fe8cb-d4b1-49a4-8eaa-6b7fea2619ce None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:18.876649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c83a35-6fb6-475c-8a71-e1c181c9e459 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:18.876923 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c83a35-6fb6-475c-8a71-e1c181c9e459 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:18.877185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c83a35-6fb6-475c-8a71-e1c181c9e459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:18.899370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c83a35-6fb6-475c-8a71-e1c181c9e459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:18.901122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c83a35-6fb6-475c-8a71-e1c181c9e459 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a4c83a35-6fb6-475c-8a71-e1c181c9e459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:44:18.931770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:18.932622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:18.933421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:44:18.933421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:44:18.933421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:44:18.933421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:18.935814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 13/28] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:44:18.937639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c83a35-6fb6-475c-8a71-e1c181c9e459 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:19.040341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fe8cb-d4b1-49a4-8eaa-6b7fea2619ce None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:19.119719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c83a35-6fb6-475c-8a71-e1c181c9e459 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:19.126291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f505dfc-45e8-486a-a287-877beafa24ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:19.126993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f505dfc-45e8-486a-a287-877beafa24ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:19.127281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f505dfc-45e8-486a-a287-877beafa24ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:19.127556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f505dfc-45e8-486a-a287-877beafa24ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f505dfc-45e8-486a-a287-877beafa24ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3f505dfc-45e8-486a-a287-877beafa24ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:19.158462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:19.159323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:19.160115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 16/29] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:44:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:19.160839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f505dfc-45e8-486a-a287-877beafa24ad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:19.179680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11787fa-79d7-4c23-b06c-f33960c7fd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:19.180645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11787fa-79d7-4c23-b06c-f33960c7fd3e None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:19.181144 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11787fa-79d7-4c23-b06c-f33960c7fd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:19.181627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11787fa-79d7-4c23-b06c-f33960c7fd3e None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:19.182570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11787fa-79d7-4c23-b06c-f33960c7fd3e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:19.184163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11787fa-79d7-4c23-b06c-f33960c7fd3e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:19.200251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:19.303730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f505dfc-45e8-486a-a287-877beafa24ad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:19.332415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 14/30] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Jul 30 04:44:18 2022] POST /identity/v3/roles => generated 219 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:19.333883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11787fa-79d7-4c23-b06c-f33960c7fd3e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:19.346782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:19.464764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa10dd-8518-4262-9530-c7ea856749b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:19.465885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa10dd-8518-4262-9530-c7ea856749b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:19.465885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa10dd-8518-4262-9530-c7ea856749b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:19.466062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa10dd-8518-4262-9530-c7ea856749b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:19.466702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa10dd-8518-4262-9530-c7ea856749b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:19.467009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa10dd-8518-4262-9530-c7ea856749b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:19.469176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa10dd-8518-4262-9530-c7ea856749b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:19.494037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11787fa-79d7-4c23-b06c-f33960c7fd3e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:19.504762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 17/31] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:19.505837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2c9bd-5e4e-485a-8f92-0ec7d592ecf9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:19.506131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2c9bd-5e4e-485a-8f92-0ec7d592ecf9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:19.506408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2c9bd-5e4e-485a-8f92-0ec7d592ecf9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:19.507845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 18/32] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:19.516194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e036590-d030-422f-bed5-fbd954d4936c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:19.516487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e036590-d030-422f-bed5-fbd954d4936c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:19.516760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e036590-d030-422f-bed5-fbd954d4936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:19.538218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47322f8e-8fd6-4bc4-a161-cc733c11d0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:19.539867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47322f8e-8fd6-4bc4-a161-cc733c11d0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:19.540367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47322f8e-8fd6-4bc4-a161-cc733c11d0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:19.540839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47322f8e-8fd6-4bc4-a161-cc733c11d0e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:19.541800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47322f8e-8fd6-4bc4-a161-cc733c11d0e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:19.542320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47322f8e-8fd6-4bc4-a161-cc733c11d0e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:19.544278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47322f8e-8fd6-4bc4-a161-cc733c11d0e3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:19.555658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0e036590-d030-422f-bed5-fbd954d4936c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:19.556830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e036590-d030-422f-bed5-fbd954d4936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:19.561088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 15/33] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:19.562362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47322f8e-8fd6-4bc4-a161-cc733c11d0e3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:19.666613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 19/34] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:19.667266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e036590-d030-422f-bed5-fbd954d4936c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:19.669249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:19.677571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:19.728101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47322f8e-8fd6-4bc4-a161-cc733c11d0e3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:19.781543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828c9b8-541e-4b70-a7ca-bd37dd15cced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:19.782527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828c9b8-541e-4b70-a7ca-bd37dd15cced None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:19.783283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828c9b8-541e-4b70-a7ca-bd37dd15cced None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:19.783739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828c9b8-541e-4b70-a7ca-bd37dd15cced None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:19.784680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828c9b8-541e-4b70-a7ca-bd37dd15cced None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:19.787546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828c9b8-541e-4b70-a7ca-bd37dd15cced None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:19.814807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e036590-d030-422f-bed5-fbd954d4936c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:19.839779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 16/35] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:44:19 2022] POST /identity/v3/projects => generated 301 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:19.840744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828c9b8-541e-4b70-a7ca-bd37dd15cced None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:19.893446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec063cbb-98bb-467b-8b8e-51bb178f792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:19.894505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec063cbb-98bb-467b-8b8e-51bb178f792d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:19.894795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec063cbb-98bb-467b-8b8e-51bb178f792d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:19.895072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec063cbb-98bb-467b-8b8e-51bb178f792d None admin] PATH_INFO: `/v3/users/74c09f97e34c428a84e13d34f5105ca9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:19.899895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec063cbb-98bb-467b-8b8e-51bb178f792d None admin] RBAC: Authorizing `identity:get_user(user_id=74c09f97e34c428a84e13d34f5105ca9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:19.904202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec063cbb-98bb-467b-8b8e-51bb178f792d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:19.906361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 20/36] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:19 2022] GET /identity/v3/users/74c09f97e34c428a84e13d34f5105ca9 => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:19.915517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec063cbb-98bb-467b-8b8e-51bb178f792d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:20.051342 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828c9b8-541e-4b70-a7ca-bd37dd15cced None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:20.073470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:20.081043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec063cbb-98bb-467b-8b8e-51bb178f792d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:20.090397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:20.091427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7b2c-3bbf-47e3-aced-ac79be82ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:20.093205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7b2c-3bbf-47e3-aced-ac79be82ab5b None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.093789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7b2c-3bbf-47e3-aced-ac79be82ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.094645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7b2c-3bbf-47e3-aced-ac79be82ab5b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.095677 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7b2c-3bbf-47e3-aced-ac79be82ab5b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:20.097491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7b2c-3bbf-47e3-aced-ac79be82ab5b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:20.117543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 17/37] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Jul 30 04:44:19 2022] POST /identity/v3/roles => generated 225 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:20.131687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a68e7-2a2e-48cf-b02d-6328aa7e7a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:20.132737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a68e7-2a2e-48cf-b02d-6328aa7e7a19 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.133238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a68e7-2a2e-48cf-b02d-6328aa7e7a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.133709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a68e7-2a2e-48cf-b02d-6328aa7e7a19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.136408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a68e7-2a2e-48cf-b02d-6328aa7e7a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:20.138205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a68e7-2a2e-48cf-b02d-6328aa7e7a19 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:20.140761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 21/38] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:20.147351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7b2c-3bbf-47e3-aced-ac79be82ab5b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:20.175636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf52f59-82bb-4303-bc1b-df251e466b45 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.176251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf52f59-82bb-4303-bc1b-df251e466b45 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.176729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf52f59-82bb-4303-bc1b-df251e466b45 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.178805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 22/39] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:20.183985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca0971-916e-4a06-93b5-8e95df9cdd07 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.184669 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca0971-916e-4a06-93b5-8e95df9cdd07 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.185144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca0971-916e-4a06-93b5-8e95df9cdd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.210396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bfca0971-916e-4a06-93b5-8e95df9cdd07 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:20.211708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bfca0971-916e-4a06-93b5-8e95df9cdd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:20.269541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7b2c-3bbf-47e3-aced-ac79be82ab5b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:20.292710 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 23/40] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:20.299871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606dc48-20f4-42b3-bc94-e4a1b196aff2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.300166 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606dc48-20f4-42b3-bc94-e4a1b196aff2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.300446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606dc48-20f4-42b3-bc94-e4a1b196aff2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.301781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 24/41] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:20.306816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d863c98-6e9a-42cc-b824-82f4d5fcb242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:20.307069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e675-5eb6-4556-a1fd-55158cf78c25 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.307311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e675-5eb6-4556-a1fd-55158cf78c25 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.307493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d863c98-6e9a-42cc-b824-82f4d5fcb242 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.307587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e675-5eb6-4556-a1fd-55158cf78c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.307772 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d863c98-6e9a-42cc-b824-82f4d5fcb242 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.308048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d863c98-6e9a-42cc-b824-82f4d5fcb242 None admin] PATH_INFO: `/v3/domains/default/users/74c09f97e34c428a84e13d34f5105ca9/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.310073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d863c98-6e9a-42cc-b824-82f4d5fcb242 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=74c09f97e34c428a84e13d34f5105ca9, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:20.312104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d863c98-6e9a-42cc-b824-82f4d5fcb242 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:20.331097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 18/42] 10.222.0.20 () {58 vars in 1361 bytes} [Sat Jul 30 04:44:20 2022] PUT /identity/v3/domains/default/users/74c09f97e34c428a84e13d34f5105ca9/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:44:20.331699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d863c98-6e9a-42cc-b824-82f4d5fcb242 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:20.332932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-87c5e675-5eb6-4556-a1fd-55158cf78c25 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:20.334161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-87c5e675-5eb6-4556-a1fd-55158cf78c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:20.388606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:20.434887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 25/43] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:20.459264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5e675-5eb6-4556-a1fd-55158cf78c25 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:20.461464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:20.478577 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d863c98-6e9a-42cc-b824-82f4d5fcb242 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:20.519063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a48a5b-036b-4c31-9d6b-55c9daa42b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:20.520413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a48a5b-036b-4c31-9d6b-55c9daa42b22 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.520865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a48a5b-036b-4c31-9d6b-55c9daa42b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.521314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a48a5b-036b-4c31-9d6b-55c9daa42b22 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.522167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a48a5b-036b-4c31-9d6b-55c9daa42b22 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:20.523750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a48a5b-036b-4c31-9d6b-55c9daa42b22 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:20.540015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 19/44] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Jul 30 04:44:20 2022] POST /identity/v3/roles => generated 223 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:20.579038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5e675-5eb6-4556-a1fd-55158cf78c25 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:20.621066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e5b77-4285-4dab-ac25-331ae088865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:20.621944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e5b77-4285-4dab-ac25-331ae088865a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.622374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e5b77-4285-4dab-ac25-331ae088865a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.622808 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e5b77-4285-4dab-ac25-331ae088865a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.625260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e5b77-4285-4dab-ac25-331ae088865a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:20.626968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e5b77-4285-4dab-ac25-331ae088865a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:20.628815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 26/45] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:20.634998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a48a5b-036b-4c31-9d6b-55c9daa42b22 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:20.751228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:20.753704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a48a5b-036b-4c31-9d6b-55c9daa42b22 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:20.789687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c3054-0eb8-458a-bea0-7974ecc86006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:20.790318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c3054-0eb8-458a-bea0-7974ecc86006 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:20.790638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c3054-0eb8-458a-bea0-7974ecc86006 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:20.790935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c3054-0eb8-458a-bea0-7974ecc86006 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:20.791620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c3054-0eb8-458a-bea0-7974ecc86006 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:20.793077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c3054-0eb8-458a-bea0-7974ecc86006 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:20.819878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 20/46] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:44:20 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:22.359878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040bfbc-c397-4f51-8b4a-101bcf3861e4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:22.360385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040bfbc-c397-4f51-8b4a-101bcf3861e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:22.360385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040bfbc-c397-4f51-8b4a-101bcf3861e4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:22.361706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 27/47] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:22.366926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e80e45-a29f-4628-9ef9-f7599a4b6bc8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:22.367336 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e80e45-a29f-4628-9ef9-f7599a4b6bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:22.367498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e80e45-a29f-4628-9ef9-f7599a4b6bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:22.385634 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c0e80e45-a29f-4628-9ef9-f7599a4b6bc8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:22.387204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c0e80e45-a29f-4628-9ef9-f7599a4b6bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:22.453382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 21/48] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:22.476376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040bfbc-c397-4f51-8b4a-101bcf3861e4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:22.548150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:22.581267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040bfbc-c397-4f51-8b4a-101bcf3861e4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:22.615663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a490e5b-b977-4421-9206-0e208fb72353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:22.616278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a490e5b-b977-4421-9206-0e208fb72353 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:22.616561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a490e5b-b977-4421-9206-0e208fb72353 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:22.616834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a490e5b-b977-4421-9206-0e208fb72353 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:22.627616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a490e5b-b977-4421-9206-0e208fb72353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1a490e5b-b977-4421-9206-0e208fb72353 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:22.632616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:22.633625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:22.634653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 28/49] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:44:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:22.640151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e80e45-a29f-4628-9ef9-f7599a4b6bc8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:22.720747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:22.740679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e80e45-a29f-4628-9ef9-f7599a4b6bc8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:22.774222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dcc7e-7b61-4484-9096-3f073b70e5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:22.774872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dcc7e-7b61-4484-9096-3f073b70e5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:22.775409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dcc7e-7b61-4484-9096-3f073b70e5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:22.775691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dcc7e-7b61-4484-9096-3f073b70e5c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:22.776418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dcc7e-7b61-4484-9096-3f073b70e5c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:22.776727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dcc7e-7b61-4484-9096-3f073b70e5c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:22.778131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dcc7e-7b61-4484-9096-3f073b70e5c8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:22.786235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 22/50] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:22.791703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a490e5b-b977-4421-9206-0e208fb72353 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:22.837382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c6f66-b20f-43b3-9352-21f1d744237a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:22.837670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c6f66-b20f-43b3-9352-21f1d744237a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:22.837947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c6f66-b20f-43b3-9352-21f1d744237a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:22.839337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 23/51] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:22.844397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fe7ab-7636-4dba-843b-1d9e33f695c7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:22.844685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fe7ab-7636-4dba-843b-1d9e33f695c7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:22.844968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fe7ab-7636-4dba-843b-1d9e33f695c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:22.864136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e82fe7ab-7636-4dba-843b-1d9e33f695c7 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:22.865353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e82fe7ab-7636-4dba-843b-1d9e33f695c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:22.893539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a490e5b-b977-4421-9206-0e208fb72353 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:22.922996 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:22.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f32f33-913b-44ff-beaa-cbd300f2bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:22.930972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f32f33-913b-44ff-beaa-cbd300f2bc38 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:22.931249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f32f33-913b-44ff-beaa-cbd300f2bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:22.931515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f32f33-913b-44ff-beaa-cbd300f2bc38 None admin] PATH_INFO: `/v3/users/74c09f97e34c428a84e13d34f5105ca9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:22.932666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f32f33-913b-44ff-beaa-cbd300f2bc38 None admin] RBAC: Authorizing `identity:get_user(user_id=74c09f97e34c428a84e13d34f5105ca9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:22.932980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 24/52] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:22.934111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f32f33-913b-44ff-beaa-cbd300f2bc38 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:22.935749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 29/53] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:22 2022] GET /identity/v3/users/74c09f97e34c428a84e13d34f5105ca9 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:22.941676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fe7ab-7636-4dba-843b-1d9e33f695c7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:22.956378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f32f33-913b-44ff-beaa-cbd300f2bc38 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:23.016659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:23.043126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fe7ab-7636-4dba-843b-1d9e33f695c7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:23.058153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f32f33-913b-44ff-beaa-cbd300f2bc38 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:23.077090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e3e36-9ac9-474f-b9d3-860dd8dc9576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:23.077925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e3e36-9ac9-474f-b9d3-860dd8dc9576 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:23.078211 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e3e36-9ac9-474f-b9d3-860dd8dc9576 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:23.078526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e3e36-9ac9-474f-b9d3-860dd8dc9576 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:23.079762 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e3e36-9ac9-474f-b9d3-860dd8dc9576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:23.081202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e3e36-9ac9-474f-b9d3-860dd8dc9576 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:23.083016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 25/54] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:23.088674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e3e36-9ac9-474f-b9d3-860dd8dc9576 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:23.092322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c880c-76cb-4acd-b647-632151edfd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:23.093537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c880c-76cb-4acd-b647-632151edfd66 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:23.093788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c880c-76cb-4acd-b647-632151edfd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:23.094067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c880c-76cb-4acd-b647-632151edfd66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:23.095281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c880c-76cb-4acd-b647-632151edfd66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:23.096733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c880c-76cb-4acd-b647-632151edfd66 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:23.098333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 30/55] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:23.104381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c880c-76cb-4acd-b647-632151edfd66 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:23.170975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:23.187843 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e3e36-9ac9-474f-b9d3-860dd8dc9576 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:23.206331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c880c-76cb-4acd-b647-632151edfd66 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:23.221867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb558c8-344d-4f38-be44-6ba03346ba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:23.222515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb558c8-344d-4f38-be44-6ba03346ba54 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:23.222802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb558c8-344d-4f38-be44-6ba03346ba54 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:23.223078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb558c8-344d-4f38-be44-6ba03346ba54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:23.223867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb558c8-344d-4f38-be44-6ba03346ba54 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.domain.id=default, user.id=74c09f97e34c428a84e13d34f5105ca9) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:23.224177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb558c8-344d-4f38-be44-6ba03346ba54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:23.225615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb558c8-344d-4f38-be44-6ba03346ba54 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:23.232638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 26/56] 10.222.0.20 () {56 vars in 1322 bytes} [Sat Jul 30 04:44:23 2022] GET /identity/v3/role_assignments?user.id=74c09f97e34c428a84e13d34f5105ca9&scope.domain.id=default&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:23.240975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b8c0e-08c5-435a-806e-caa7bd159420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:23.241618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b8c0e-08c5-435a-806e-caa7bd159420 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:23.241899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b8c0e-08c5-435a-806e-caa7bd159420 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:23.242173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b8c0e-08c5-435a-806e-caa7bd159420 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:23.242905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b8c0e-08c5-435a-806e-caa7bd159420 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:23.244404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b8c0e-08c5-435a-806e-caa7bd159420 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:23.390293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 31/57] 10.222.0.20 () {60 vars in 1140 bytes} [Sat Jul 30 04:44:23 2022] POST /identity/v3/projects => generated 298 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:44:25.389413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e8648-66f5-4e0f-bfe3-635bb3a65d3c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:25.389928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e8648-66f5-4e0f-bfe3-635bb3a65d3c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:25.389928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e8648-66f5-4e0f-bfe3-635bb3a65d3c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:25.391882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 27/58] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:25.397200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93914c-ae26-46a0-830c-e13ff75f0977 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:25.397631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93914c-ae26-46a0-830c-e13ff75f0977 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:25.397761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93914c-ae26-46a0-830c-e13ff75f0977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:25.416388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1b93914c-ae26-46a0-830c-e13ff75f0977 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:25.417894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1b93914c-ae26-46a0-830c-e13ff75f0977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:25.487590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 32/59] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:25.511883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e8648-66f5-4e0f-bfe3-635bb3a65d3c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:25.521471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:25.610835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e8648-66f5-4e0f-bfe3-635bb3a65d3c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:25.643775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e57d72-1147-46da-9228-eebe0b44b0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:25.644342 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57d72-1147-46da-9228-eebe0b44b0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:25.644620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57d72-1147-46da-9228-eebe0b44b0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:25.644890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57d72-1147-46da-9228-eebe0b44b0c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:25.646022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e57d72-1147-46da-9228-eebe0b44b0c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:25.647500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e57d72-1147-46da-9228-eebe0b44b0c3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:25.649057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 28/60] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:25.654781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93914c-ae26-46a0-830c-e13ff75f0977 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:25.704127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:25.755701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93914c-ae26-46a0-830c-e13ff75f0977 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:25.788425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cff9c-39d2-4b24-96e4-fd1e79de6c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:25.789043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cff9c-39d2-4b24-96e4-fd1e79de6c4b None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:25.789329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cff9c-39d2-4b24-96e4-fd1e79de6c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:25.789604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cff9c-39d2-4b24-96e4-fd1e79de6c4b None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:25.790299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cff9c-39d2-4b24-96e4-fd1e79de6c4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:25.791908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cff9c-39d2-4b24-96e4-fd1e79de6c4b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:25.817902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 33/61] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:44:25 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:44:28.779672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7c042-ecfc-458c-b28a-1e6a64d460e7 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:28.781181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7c042-ecfc-458c-b28a-1e6a64d460e7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:28.781667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7c042-ecfc-458c-b28a-1e6a64d460e7 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:28.783274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 29/62] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:28.788622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99b3bf-cc6b-4134-bfa7-2f1f46d363df None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:28.789445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99b3bf-cc6b-4134-bfa7-2f1f46d363df None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:28.789917 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99b3bf-cc6b-4134-bfa7-2f1f46d363df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:28.809914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4e99b3bf-cc6b-4134-bfa7-2f1f46d363df None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:28.811612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e99b3bf-cc6b-4134-bfa7-2f1f46d363df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:28.884206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 34/63] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:28.909288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7c042-ecfc-458c-b28a-1e6a64d460e7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:29.010800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:29.041515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7c042-ecfc-458c-b28a-1e6a64d460e7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:29.089590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537dad5-50ab-4897-988a-652e5458c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:29.090245 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537dad5-50ab-4897-988a-652e5458c9cd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.090549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537dad5-50ab-4897-988a-652e5458c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.090822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537dad5-50ab-4897-988a-652e5458c9cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.107778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537dad5-50ab-4897-988a-652e5458c9cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c537dad5-50ab-4897-988a-652e5458c9cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:29.114252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:29.115165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:29.115927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 30/64] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:29.122465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99b3bf-cc6b-4134-bfa7-2f1f46d363df None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:29.187403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:29.223353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd997b6-c9ea-439e-b1ea-3d5ef2d73fde None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.223963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd997b6-c9ea-439e-b1ea-3d5ef2d73fde None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.224470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd997b6-c9ea-439e-b1ea-3d5ef2d73fde None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.226035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 31/65] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:29.231388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d078f-f80e-4ad2-9aaf-ca1c2376a9f4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.231909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d078f-f80e-4ad2-9aaf-ca1c2376a9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.232386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d078f-f80e-4ad2-9aaf-ca1c2376a9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.238167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99b3bf-cc6b-4134-bfa7-2f1f46d363df None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:29.262854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d20d078f-f80e-4ad2-9aaf-ca1c2376a9f4 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:29.264475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d20d078f-f80e-4ad2-9aaf-ca1c2376a9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:29.275187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba065e-cdca-40aa-b371-bed0d3c63b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:29.276068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba065e-cdca-40aa-b371-bed0d3c63b32 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.276355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba065e-cdca-40aa-b371-bed0d3c63b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.276633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba065e-cdca-40aa-b371-bed0d3c63b32 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.277350 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba065e-cdca-40aa-b371-bed0d3c63b32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:29.277655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba065e-cdca-40aa-b371-bed0d3c63b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:29.279155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba065e-cdca-40aa-b371-bed0d3c63b32 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:29.291014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 35/66] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:29.296778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e7def-b1db-4e5d-9b79-cd2d6e1129d4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.297304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e7def-b1db-4e5d-9b79-cd2d6e1129d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.297740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e7def-b1db-4e5d-9b79-cd2d6e1129d4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.299366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 36/67] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:29.300163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e7def-b1db-4e5d-9b79-cd2d6e1129d4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:29.355602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 32/68] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:29.357063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29080e13-cade-4154-8eb3-729c2e10bf85 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.357489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29080e13-cade-4154-8eb3-729c2e10bf85 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.357883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29080e13-cade-4154-8eb3-729c2e10bf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.380067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-29080e13-cade-4154-8eb3-729c2e10bf85 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:29.381752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-29080e13-cade-4154-8eb3-729c2e10bf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:29.432781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e7def-b1db-4e5d-9b79-cd2d6e1129d4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:29.445402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:29.450845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:29.464439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 33/69] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:29.470249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29080e13-cade-4154-8eb3-729c2e10bf85 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:29.478464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10974521-4742-4e05-8537-984bd375cca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:29.478464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974521-4742-4e05-8537-984bd375cca6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.478464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974521-4742-4e05-8537-984bd375cca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.478464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974521-4742-4e05-8537-984bd375cca6 None admin] PATH_INFO: `/v3/users/5716eec426c840c79a2d8f3c7ec7ad18` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.493850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10974521-4742-4e05-8537-984bd375cca6 None admin] RBAC: Authorizing `identity:get_user(user_id=5716eec426c840c79a2d8f3c7ec7ad18)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:29.495941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10974521-4742-4e05-8537-984bd375cca6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:29.497941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 37/70] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:29 2022] GET /identity/v3/users/5716eec426c840c79a2d8f3c7ec7ad18 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:29.502379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10974521-4742-4e05-8537-984bd375cca6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:29.604393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29080e13-cade-4154-8eb3-729c2e10bf85 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:29.619607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10974521-4742-4e05-8537-984bd375cca6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:29.637961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf696b3-b521-457c-a740-851bd57e41d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:29.638831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf696b3-b521-457c-a740-851bd57e41d7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.639120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf696b3-b521-457c-a740-851bd57e41d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.639399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf696b3-b521-457c-a740-851bd57e41d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.650757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf696b3-b521-457c-a740-851bd57e41d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:29.655991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114094b-ac30-42d6-8a4d-633f9d57d485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1cf696b3-b521-457c-a740-851bd57e41d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:29.658381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:29.659258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:29.659996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:29.660568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114094b-ac30-42d6-8a4d-633f9d57d485 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.660568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114094b-ac30-42d6-8a4d-633f9d57d485 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.660568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114094b-ac30-42d6-8a4d-633f9d57d485 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.660737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 34/71] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:44:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:29.660737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf696b3-b521-457c-a740-851bd57e41d7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:29.673602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114094b-ac30-42d6-8a4d-633f9d57d485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f114094b-ac30-42d6-8a4d-633f9d57d485 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:29.678699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:29.679774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:29.680541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 38/72] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:44:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:44:29.681471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07157485-1bc4-45e4-8230-724ac4a1d203 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.681784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07157485-1bc4-45e4-8230-724ac4a1d203 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.682036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07157485-1bc4-45e4-8230-724ac4a1d203 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.683319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 39/73] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:29.687804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01798b-8609-4dc2-840a-95b2416bfb35 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.688089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01798b-8609-4dc2-840a-95b2416bfb35 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.688368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01798b-8609-4dc2-840a-95b2416bfb35 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.689697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 40/74] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:29.690403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01798b-8609-4dc2-840a-95b2416bfb35 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:29.706378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:29.724880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:29.770902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf696b3-b521-457c-a740-851bd57e41d7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:29.795903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01798b-8609-4dc2-840a-95b2416bfb35 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:29.804424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c8fe7-fb4d-4c23-b0ab-08984bab6aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:29.805037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c8fe7-fb4d-4c23-b0ab-08984bab6aec None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.805324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c8fe7-fb4d-4c23-b0ab-08984bab6aec None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.805597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c8fe7-fb4d-4c23-b0ab-08984bab6aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.831972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4b266-2754-469d-ae84-40a104ba93e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:29.832601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4b266-2754-469d-ae84-40a104ba93e2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.832886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4b266-2754-469d-ae84-40a104ba93e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.833183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4b266-2754-469d-ae84-40a104ba93e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.833889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4b266-2754-469d-ae84-40a104ba93e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:29.834198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4b266-2754-469d-ae84-40a104ba93e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:29.835640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4b266-2754-469d-ae84-40a104ba93e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:29.843534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 41/75] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:44:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:29.844325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4b266-2754-469d-ae84-40a104ba93e2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:29.877301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c8fe7-fb4d-4c23-b0ab-08984bab6aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:29.878717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c8fe7-fb4d-4c23-b0ab-08984bab6aec None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:29.920550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:29.932743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:29.946379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4b266-2754-469d-ae84-40a104ba93e2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:29.981387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7a8e7-4def-40ca-8777-88431ed9e5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:29.982014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7a8e7-4def-40ca-8777-88431ed9e5fe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.982296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7a8e7-4def-40ca-8777-88431ed9e5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.982600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7a8e7-4def-40ca-8777-88431ed9e5fe None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.983616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7a8e7-4def-40ca-8777-88431ed9e5fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:29.984195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7a8e7-4def-40ca-8777-88431ed9e5fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:29.984309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 35/76] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:29.985506 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb384e-4907-45c2-949d-1a675e69d4f5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.985605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7a8e7-4def-40ca-8777-88431ed9e5fe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:29.985795 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb384e-4907-45c2-949d-1a675e69d4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:29.986072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb384e-4907-45c2-949d-1a675e69d4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:29.994236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 42/77] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:29.999591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c0e3a-f8ab-4570-b6f5-ba299ccd4c1a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:29.999890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c0e3a-f8ab-4570-b6f5-ba299ccd4c1a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.000175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c0e3a-f8ab-4570-b6f5-ba299ccd4c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.004331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2ffb384e-4907-45c2-949d-1a675e69d4f5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:30.005535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2ffb384e-4907-45c2-949d-1a675e69d4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:30.019077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f13c0e3a-f8ab-4570-b6f5-ba299ccd4c1a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:30.021125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f13c0e3a-f8ab-4570-b6f5-ba299ccd4c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:30.073492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 36/78] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:30.074340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb384e-4907-45c2-949d-1a675e69d4f5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.088388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 43/79] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:30.089051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c0e3a-f8ab-4570-b6f5-ba299ccd4c1a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.160039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:30.162173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:30.177244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb384e-4907-45c2-949d-1a675e69d4f5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:30.196853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c0e3a-f8ab-4570-b6f5-ba299ccd4c1a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:30.211485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22535f-98a5-4837-afca-db4604825f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:30.212139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22535f-98a5-4837-afca-db4604825f80 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:30.212422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22535f-98a5-4837-afca-db4604825f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.212760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22535f-98a5-4837-afca-db4604825f80 None admin] PATH_INFO: `/v3/users/5716eec426c840c79a2d8f3c7ec7ad18` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.213987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22535f-98a5-4837-afca-db4604825f80 None admin] RBAC: Authorizing `identity:get_user(user_id=5716eec426c840c79a2d8f3c7ec7ad18)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:30.215466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22535f-98a5-4837-afca-db4604825f80 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:30.217082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 37/80] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:30 2022] GET /identity/v3/users/5716eec426c840c79a2d8f3c7ec7ad18 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:30.217082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22535f-98a5-4837-afca-db4604825f80 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.231805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e2a33-33f1-439e-9515-ab078e0a46ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:30.232751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e2a33-33f1-439e-9515-ab078e0a46ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:30.233056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e2a33-33f1-439e-9515-ab078e0a46ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.233861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e2a33-33f1-439e-9515-ab078e0a46ac None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.235086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e2a33-33f1-439e-9515-ab078e0a46ac None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:30.236545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e2a33-33f1-439e-9515-ab078e0a46ac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:30.238209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 44/81] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:30 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:30.243269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e2a33-33f1-439e-9515-ab078e0a46ac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.320249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22535f-98a5-4837-afca-db4604825f80 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:30.344119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e2a33-33f1-439e-9515-ab078e0a46ac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:30.353818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe742b-6675-436a-b26c-bd17b440a8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:30.354484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe742b-6675-436a-b26c-bd17b440a8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:30.354771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe742b-6675-436a-b26c-bd17b440a8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.355047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe742b-6675-436a-b26c-bd17b440a8b0 None admin] PATH_INFO: `/v3/users/74c09f97e34c428a84e13d34f5105ca9` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.356233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe742b-6675-436a-b26c-bd17b440a8b0 None admin] RBAC: Authorizing `identity:get_user(user_id=74c09f97e34c428a84e13d34f5105ca9)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:30.357733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe742b-6675-436a-b26c-bd17b440a8b0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:30.359890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 38/82] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:30 2022] GET /identity/v3/users/74c09f97e34c428a84e13d34f5105ca9 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:30.360652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe742b-6675-436a-b26c-bd17b440a8b0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.382047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:30.382951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae42b97-25ea-4241-ab4e-59b5d7a52324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:30.383543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae42b97-25ea-4241-ab4e-59b5d7a52324 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:30.383824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae42b97-25ea-4241-ab4e-59b5d7a52324 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.384095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae42b97-25ea-4241-ab4e-59b5d7a52324 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.385251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae42b97-25ea-4241-ab4e-59b5d7a52324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:30.386722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae42b97-25ea-4241-ab4e-59b5d7a52324 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:30.388300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 45/83] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:30.389043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae42b97-25ea-4241-ab4e-59b5d7a52324 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.396031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:30.461622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe742b-6675-436a-b26c-bd17b440a8b0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:30.493310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae42b97-25ea-4241-ab4e-59b5d7a52324 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:30.494867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26126544-5ea7-4b5f-ac3e-79634c360d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:30.495500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26126544-5ea7-4b5f-ac3e-79634c360d44 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:30.495780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26126544-5ea7-4b5f-ac3e-79634c360d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.496045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26126544-5ea7-4b5f-ac3e-79634c360d44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.506854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26126544-5ea7-4b5f-ac3e-79634c360d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-26126544-5ea7-4b5f-ac3e-79634c360d44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:30.511742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:30.512727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:30.513507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 39/84] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:30.514140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26126544-5ea7-4b5f-ac3e-79634c360d44 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.532256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a385f1-6ddd-42c0-9c05-40154dc291e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:30.532880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a385f1-6ddd-42c0-9c05-40154dc291e2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:30.533163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a385f1-6ddd-42c0-9c05-40154dc291e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.533436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a385f1-6ddd-42c0-9c05-40154dc291e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.596716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:30.601830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a385f1-6ddd-42c0-9c05-40154dc291e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:30.603358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a385f1-6ddd-42c0-9c05-40154dc291e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:30.605914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:30.616822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26126544-5ea7-4b5f-ac3e-79634c360d44 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:30.651360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63269479-2e54-41fa-966a-a822ead2091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:30.652176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63269479-2e54-41fa-966a-a822ead2091d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:30.652454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63269479-2e54-41fa-966a-a822ead2091d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.652728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63269479-2e54-41fa-966a-a822ead2091d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.653521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63269479-2e54-41fa-966a-a822ead2091d None admin] RBAC: Adding query filter params (role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=395264e10f8345d89e730d4c1a103dd2, user.id=5716eec426c840c79a2d8f3c7ec7ad18) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:30.653831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63269479-2e54-41fa-966a-a822ead2091d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:30.655886 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63269479-2e54-41fa-966a-a822ead2091d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:30.670471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 40/85] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:30 2022] GET /identity/v3/role_assignments?user.id=5716eec426c840c79a2d8f3c7ec7ad18&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:30.670471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63269479-2e54-41fa-966a-a822ead2091d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.708069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 46/86] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:30.708941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a385f1-6ddd-42c0-9c05-40154dc291e2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.780511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63269479-2e54-41fa-966a-a822ead2091d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:30.822941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfeef2-fc62-4ed8-adbe-73f041e5f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:30.824230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfeef2-fc62-4ed8-adbe-73f041e5f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:30.824730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfeef2-fc62-4ed8-adbe-73f041e5f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.825008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfeef2-fc62-4ed8-adbe-73f041e5f6c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.832759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a385f1-6ddd-42c0-9c05-40154dc291e2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:30.836091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:30.842744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:30.879454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ae92e-f650-4281-b619-c545f5255e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:30.880416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ae92e-f650-4281-b619-c545f5255e9c None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:30.880712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ae92e-f650-4281-b619-c545f5255e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:30.881156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ae92e-f650-4281-b619-c545f5255e9c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:30.881850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ae92e-f650-4281-b619-c545f5255e9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:30.883378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ae92e-f650-4281-b619-c545f5255e9c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:30.907430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 47/87] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:44:30 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:30.908276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ae92e-f650-4281-b619-c545f5255e9c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:30.908762 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfeef2-fc62-4ed8-adbe-73f041e5f6c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:30.910339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfeef2-fc62-4ed8-adbe-73f041e5f6c6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:31.026070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ae92e-f650-4281-b619-c545f5255e9c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:31.037162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 41/88] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:31.038377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfeef2-fc62-4ed8-adbe-73f041e5f6c6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:31.070362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb41bf-9c97-4cd9-861d-051f69e74d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:31.071037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb41bf-9c97-4cd9-861d-051f69e74d0a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:31.071325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb41bf-9c97-4cd9-861d-051f69e74d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:31.071602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb41bf-9c97-4cd9-861d-051f69e74d0a None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:31.072403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb41bf-9c97-4cd9-861d-051f69e74d0a None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:31.072712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb41bf-9c97-4cd9-861d-051f69e74d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:31.074102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb41bf-9c97-4cd9-861d-051f69e74d0a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:31.084407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 48/89] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:31.085084 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb41bf-9c97-4cd9-861d-051f69e74d0a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:31.093262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:31.115212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:31.163839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfeef2-fc62-4ed8-adbe-73f041e5f6c6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:31.216099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c77358-2f0b-4605-b347-2a15e45a7116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:31.216789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c77358-2f0b-4605-b347-2a15e45a7116 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:31.217067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c77358-2f0b-4605-b347-2a15e45a7116 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:31.217340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c77358-2f0b-4605-b347-2a15e45a7116 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:31.219328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c77358-2f0b-4605-b347-2a15e45a7116 None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:31.220841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c77358-2f0b-4605-b347-2a15e45a7116 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:31.222694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 42/90] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:31 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:31.223312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c77358-2f0b-4605-b347-2a15e45a7116 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:31.229775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb41bf-9c97-4cd9-861d-051f69e74d0a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:31.284416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f99c4-508d-44d2-ae58-7fb8054a4add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:31.285316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f99c4-508d-44d2-ae58-7fb8054a4add None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:31.285734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f99c4-508d-44d2-ae58-7fb8054a4add None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:31.286313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f99c4-508d-44d2-ae58-7fb8054a4add None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:31.288693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f99c4-508d-44d2-ae58-7fb8054a4add None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:31.290427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f99c4-508d-44d2-ae58-7fb8054a4add None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:31.292326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 49/91] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:31 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:31.293260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f99c4-508d-44d2-ae58-7fb8054a4add None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:31.355898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c77358-2f0b-4605-b347-2a15e45a7116 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:31.372110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:31.402776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff4c4-1afe-46e4-a4ca-4ea8dafa7199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:31.403440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff4c4-1afe-46e4-a4ca-4ea8dafa7199 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:31.403724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff4c4-1afe-46e4-a4ca-4ea8dafa7199 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:31.404001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff4c4-1afe-46e4-a4ca-4ea8dafa7199 None admin] PATH_INFO: `/v3/users/5716eec426c840c79a2d8f3c7ec7ad18` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:31.407996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfff4c4-1afe-46e4-a4ca-4ea8dafa7199 None admin] RBAC: Authorizing `identity:get_user(user_id=5716eec426c840c79a2d8f3c7ec7ad18)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:31.409473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfff4c4-1afe-46e4-a4ca-4ea8dafa7199 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:31.411143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 43/92] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:31 2022] GET /identity/v3/users/5716eec426c840c79a2d8f3c7ec7ad18 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:31.411915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff4c4-1afe-46e4-a4ca-4ea8dafa7199 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:31.446813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:31.461526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f99c4-508d-44d2-ae58-7fb8054a4add None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:31.529466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff4c4-1afe-46e4-a4ca-4ea8dafa7199 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:31.541556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4aa50-76f5-4570-8ee0-2ff89bc627d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:31.542110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4aa50-76f5-4570-8ee0-2ff89bc627d7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:31.542397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4aa50-76f5-4570-8ee0-2ff89bc627d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:31.542710 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4aa50-76f5-4570-8ee0-2ff89bc627d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:31.543510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4aa50-76f5-4570-8ee0-2ff89bc627d7 None admin] RBAC: Adding query filter params (role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=395264e10f8345d89e730d4c1a103dd2, user.id=5716eec426c840c79a2d8f3c7ec7ad18) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:31.543826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4aa50-76f5-4570-8ee0-2ff89bc627d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:31.545279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4aa50-76f5-4570-8ee0-2ff89bc627d7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:31.557771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 50/93] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:31 2022] GET /identity/v3/role_assignments?user.id=5716eec426c840c79a2d8f3c7ec7ad18&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:31.558457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4aa50-76f5-4570-8ee0-2ff89bc627d7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:31.631261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaba8c-c0e4-4acc-af8c-fecca5ba6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:31.631913 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaba8c-c0e4-4acc-af8c-fecca5ba6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:31.632198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaba8c-c0e4-4acc-af8c-fecca5ba6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:31.632473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaba8c-c0e4-4acc-af8c-fecca5ba6cf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:31.633285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaba8c-c0e4-4acc-af8c-fecca5ba6cf6 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=395264e10f8345d89e730d4c1a103dd2, user.id=74c09f97e34c428a84e13d34f5105ca9) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:31.633595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaba8c-c0e4-4acc-af8c-fecca5ba6cf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:31.635087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaba8c-c0e4-4acc-af8c-fecca5ba6cf6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:31.645382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 44/94] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:31 2022] GET /identity/v3/role_assignments?user.id=74c09f97e34c428a84e13d34f5105ca9&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:31.751688 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4aa50-76f5-4570-8ee0-2ff89bc627d7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:31.807988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da7d10-1d4c-4d6b-ba43-a839d5a6c224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:31.808570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da7d10-1d4c-4d6b-ba43-a839d5a6c224 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:31.808861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da7d10-1d4c-4d6b-ba43-a839d5a6c224 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:31.809142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da7d10-1d4c-4d6b-ba43-a839d5a6c224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:31.854801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:31.903993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da7d10-1d4c-4d6b-ba43-a839d5a6c224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:31.905576 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da7d10-1d4c-4d6b-ba43-a839d5a6c224 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:32.166914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 51/95] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:32.172636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaba8c-c0e4-4acc-af8c-fecca5ba6cf6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:32.279470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:32.386295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaba8c-c0e4-4acc-af8c-fecca5ba6cf6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:32.446991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aba0fa-aa3f-4fd7-9d4c-ea6653cbcdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:32.447566 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aba0fa-aa3f-4fd7-9d4c-ea6653cbcdce None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:32.447850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aba0fa-aa3f-4fd7-9d4c-ea6653cbcdce None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:32.448124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aba0fa-aa3f-4fd7-9d4c-ea6653cbcdce None admin] PATH_INFO: `/v3/projects/019c9a95e4514d839c157565d71c4500` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:32.451430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aba0fa-aa3f-4fd7-9d4c-ea6653cbcdce None admin] RBAC: Authorizing `identity:get_project(project_id=019c9a95e4514d839c157565d71c4500)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:32.453802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aba0fa-aa3f-4fd7-9d4c-ea6653cbcdce None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:32.455993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 45/96] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:32 2022] GET /identity/v3/projects/019c9a95e4514d839c157565d71c4500 => generated 312 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:32.462029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da7d10-1d4c-4d6b-ba43-a839d5a6c224 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:32.564261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:32.618613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da7d10-1d4c-4d6b-ba43-a839d5a6c224 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:32.667819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2fa1c-80df-45d1-ac37-073be8968ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:32.668450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2fa1c-80df-45d1-ac37-073be8968ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:32.668733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2fa1c-80df-45d1-ac37-073be8968ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:32.669010 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2fa1c-80df-45d1-ac37-073be8968ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:32.669811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2fa1c-80df-45d1-ac37-073be8968ec1 None admin] RBAC: Adding query filter params (role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=019c9a95e4514d839c157565d71c4500, user.id=5716eec426c840c79a2d8f3c7ec7ad18) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:32.670122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2fa1c-80df-45d1-ac37-073be8968ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:32.672338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2fa1c-80df-45d1-ac37-073be8968ec1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:32.678409 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 52/97] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:32 2022] GET /identity/v3/role_assignments?user.id=5716eec426c840c79a2d8f3c7ec7ad18&scope.project.id=019c9a95e4514d839c157565d71c4500&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:33.316376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d400b-686f-4cb7-810a-c5777b662465 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:33.316376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d400b-686f-4cb7-810a-c5777b662465 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:33.316376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d400b-686f-4cb7-810a-c5777b662465 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:33.316376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 46/98] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:33.322609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73333eb2-f817-4749-9869-b9df40af3cf9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:33.323408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73333eb2-f817-4749-9869-b9df40af3cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:33.323890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73333eb2-f817-4749-9869-b9df40af3cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:33.342879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-73333eb2-f817-4749-9869-b9df40af3cf9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:33.344926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73333eb2-f817-4749-9869-b9df40af3cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:33.436792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 53/99] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:33.471063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d400b-686f-4cb7-810a-c5777b662465 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:33.590641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d400b-686f-4cb7-810a-c5777b662465 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:33.604976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:33.628536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdac50c-be82-42a8-b6f5-9de1c49cb583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:33.629188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdac50c-be82-42a8-b6f5-9de1c49cb583 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:33.629481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdac50c-be82-42a8-b6f5-9de1c49cb583 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:33.629760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdac50c-be82-42a8-b6f5-9de1c49cb583 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:33.641240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdac50c-be82-42a8-b6f5-9de1c49cb583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2cdac50c-be82-42a8-b6f5-9de1c49cb583 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:33.646774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:33.647611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:33.648370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 47/100] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:33.654220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73333eb2-f817-4749-9869-b9df40af3cf9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:33.756589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:33.815620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73333eb2-f817-4749-9869-b9df40af3cf9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:33.861134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9445afc-5e0d-44df-9022-96cad0e21008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:33.861687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9445afc-5e0d-44df-9022-96cad0e21008 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:33.861971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9445afc-5e0d-44df-9022-96cad0e21008 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:33.862248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9445afc-5e0d-44df-9022-96cad0e21008 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:33.863005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9445afc-5e0d-44df-9022-96cad0e21008 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:33.863311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9445afc-5e0d-44df-9022-96cad0e21008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:33.864842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9445afc-5e0d-44df-9022-96cad0e21008 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:33.877570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 54/101] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:33.883191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdac50c-be82-42a8-b6f5-9de1c49cb583 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:34.039605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdac50c-be82-42a8-b6f5-9de1c49cb583 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:34.099719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b67f2-d457-46fd-97ed-fa5fd9c5d15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:34.100376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b67f2-d457-46fd-97ed-fa5fd9c5d15e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.100664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b67f2-d457-46fd-97ed-fa5fd9c5d15e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.100948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b67f2-d457-46fd-97ed-fa5fd9c5d15e None admin] PATH_INFO: `/v3/users/5716eec426c840c79a2d8f3c7ec7ad18` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.102166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b67f2-d457-46fd-97ed-fa5fd9c5d15e None admin] RBAC: Authorizing `identity:get_user(user_id=5716eec426c840c79a2d8f3c7ec7ad18)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:34.103664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b67f2-d457-46fd-97ed-fa5fd9c5d15e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:34.105445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 48/102] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:33 2022] GET /identity/v3/users/5716eec426c840c79a2d8f3c7ec7ad18 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:34.111584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9445afc-5e0d-44df-9022-96cad0e21008 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:34.252035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca060f1-e4b4-4622-9ebf-6e1d59ec3586 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.252559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca060f1-e4b4-4622-9ebf-6e1d59ec3586 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.252844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca060f1-e4b4-4622-9ebf-6e1d59ec3586 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.254342 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 49/103] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:34.258612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9445afc-5e0d-44df-9022-96cad0e21008 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:34.260721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abb109-2af7-4b87-8a93-684184b812bf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.261013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abb109-2af7-4b87-8a93-684184b812bf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.261295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abb109-2af7-4b87-8a93-684184b812bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.274489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:34.303446 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f1abb109-2af7-4b87-8a93-684184b812bf None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:34.304672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f1abb109-2af7-4b87-8a93-684184b812bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:34.317105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55f72b-77f6-4241-96db-ab2572c62ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:34.317766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55f72b-77f6-4241-96db-ab2572c62ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.318054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55f72b-77f6-4241-96db-ab2572c62ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.318332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55f72b-77f6-4241-96db-ab2572c62ba2 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.323172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55f72b-77f6-4241-96db-ab2572c62ba2 None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:34.324647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55f72b-77f6-4241-96db-ab2572c62ba2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:34.326565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 55/104] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:34 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:34.332403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55f72b-77f6-4241-96db-ab2572c62ba2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:34.382285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:34.511980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 50/105] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:34.516179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55f72b-77f6-4241-96db-ab2572c62ba2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:34.547826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abb109-2af7-4b87-8a93-684184b812bf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:34.571270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f0ee4-50ff-467a-be7f-42a15c24309e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:34.571984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f0ee4-50ff-467a-be7f-42a15c24309e None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.572275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f0ee4-50ff-467a-be7f-42a15c24309e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.572557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f0ee4-50ff-467a-be7f-42a15c24309e None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2/users/5716eec426c840c79a2d8f3c7ec7ad18/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.574720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f0ee4-50ff-467a-be7f-42a15c24309e None admin] RBAC: Authorizing `identity:create_grant(project_id=395264e10f8345d89e730d4c1a103dd2, user_id=5716eec426c840c79a2d8f3c7ec7ad18, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:34.576192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f0ee4-50ff-467a-be7f-42a15c24309e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:34.610588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 56/106] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:44:34 2022] PUT /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2/users/5716eec426c840c79a2d8f3c7ec7ad18/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:44:34.705253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e7a5e-307a-4b95-9032-1a3f3b9f563d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.706186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e7a5e-307a-4b95-9032-1a3f3b9f563d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.706996 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e7a5e-307a-4b95-9032-1a3f3b9f563d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.709861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 57/107] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:34.720389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e2c7-f0ee-4800-89bf-01e19e463b28 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.721035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e2c7-f0ee-4800-89bf-01e19e463b28 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.723088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e2c7-f0ee-4800-89bf-01e19e463b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.736892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abb109-2af7-4b87-8a93-684184b812bf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:34.769149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b917e2c7-f0ee-4800-89bf-01e19e463b28 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:34.772425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b917e2c7-f0ee-4800-89bf-01e19e463b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:34.818465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab057c6-32c8-4c3c-8d79-0a8f98da7d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:34.819484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab057c6-32c8-4c3c-8d79-0a8f98da7d3f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.820224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab057c6-32c8-4c3c-8d79-0a8f98da7d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.820688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab057c6-32c8-4c3c-8d79-0a8f98da7d3f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.836556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab057c6-32c8-4c3c-8d79-0a8f98da7d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:34.837510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-9ab057c6-32c8-4c3c-8d79-0a8f98da7d3f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:34.846805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:34.847681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:34.848443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 51/108] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:44:34.849808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecbb96-fbab-4cda-ba51-59a0a8e99ba6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.850095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecbb96-fbab-4cda-ba51-59a0a8e99ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.850374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecbb96-fbab-4cda-ba51-59a0a8e99ba6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.851844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 52/109] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:34.857024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b4959-82cd-42ee-b939-17abb32c1783 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:34.857308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b4959-82cd-42ee-b939-17abb32c1783 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:34.857585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b4959-82cd-42ee-b939-17abb32c1783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:34.880633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fb1b4959-82cd-42ee-b939-17abb32c1783 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:34.882329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fb1b4959-82cd-42ee-b939-17abb32c1783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:34.895971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 58/110] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:34.897241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e2c7-f0ee-4800-89bf-01e19e463b28 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:34.912664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:34.987024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 53/111] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:34.987918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b4959-82cd-42ee-b939-17abb32c1783 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:35.042187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e2c7-f0ee-4800-89bf-01e19e463b28 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:35.086258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece767e-5c42-4a30-afc8-ad10a06ee241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:35.087154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece767e-5c42-4a30-afc8-ad10a06ee241 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.087603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece767e-5c42-4a30-afc8-ad10a06ee241 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.088017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece767e-5c42-4a30-afc8-ad10a06ee241 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.089384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece767e-5c42-4a30-afc8-ad10a06ee241 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:35.089838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece767e-5c42-4a30-afc8-ad10a06ee241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:35.091854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece767e-5c42-4a30-afc8-ad10a06ee241 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:35.102271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 59/112] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:35.103819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e5079-9923-4afd-9e9d-c3fa4afd7e46 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.104247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e5079-9923-4afd-9e9d-c3fa4afd7e46 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.104663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e5079-9923-4afd-9e9d-c3fa4afd7e46 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.106031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 60/113] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:35.111865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e5079-9923-4afd-9e9d-c3fa4afd7e46 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:35.117594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b4959-82cd-42ee-b939-17abb32c1783 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:35.146172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:35.155846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb80cf-c922-40c9-86c4-6dc08f4e3359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:35.156815 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb80cf-c922-40c9-86c4-6dc08f4e3359 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.157163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb80cf-c922-40c9-86c4-6dc08f4e3359 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.157503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb80cf-c922-40c9-86c4-6dc08f4e3359 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.168398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb80cf-c922-40c9-86c4-6dc08f4e3359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e6bb80cf-c922-40c9-86c4-6dc08f4e3359 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:35.173811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:35.174853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:35.175598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 54/114] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:44:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:35.181126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb80cf-c922-40c9-86c4-6dc08f4e3359 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:35.205062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:35.234308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e5079-9923-4afd-9e9d-c3fa4afd7e46 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:35.274016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5b7c3-2b69-49de-8aeb-a1b7e0f4fc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:35.274966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5b7c3-2b69-49de-8aeb-a1b7e0f4fc8c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.275447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5b7c3-2b69-49de-8aeb-a1b7e0f4fc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.276147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5b7c3-2b69-49de-8aeb-a1b7e0f4fc8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.278129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5b7c3-2b69-49de-8aeb-a1b7e0f4fc8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:35.279903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5b7c3-2b69-49de-8aeb-a1b7e0f4fc8c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:35.282018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 61/115] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:35.287778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d85c60-94c1-4d07-a5be-bd596b06d03a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.288064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d85c60-94c1-4d07-a5be-bd596b06d03a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.288345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d85c60-94c1-4d07-a5be-bd596b06d03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.292314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb80cf-c922-40c9-86c4-6dc08f4e3359 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:35.309785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-26d85c60-94c1-4d07-a5be-bd596b06d03a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:35.311383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-26d85c60-94c1-4d07-a5be-bd596b06d03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:35.328623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b7294-7d01-4036-b431-3bdd9b13cc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:35.329420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b7294-7d01-4036-b431-3bdd9b13cc9f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.329707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b7294-7d01-4036-b431-3bdd9b13cc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.329984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b7294-7d01-4036-b431-3bdd9b13cc9f None admin] PATH_INFO: `/v3/users/a90b5f37d8ea4b1db948a26c451368e7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.345396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b7294-7d01-4036-b431-3bdd9b13cc9f None admin] RBAC: Authorizing `identity:get_user(user_id=a90b5f37d8ea4b1db948a26c451368e7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:35.347060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b7294-7d01-4036-b431-3bdd9b13cc9f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:35.349049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 55/116] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:35 2022] GET /identity/v3/users/a90b5f37d8ea4b1db948a26c451368e7 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:35.349699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b7294-7d01-4036-b431-3bdd9b13cc9f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:35.389461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 62/117] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:35.391578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d85c60-94c1-4d07-a5be-bd596b06d03a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:35.440590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:35.455934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:35.486867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b7294-7d01-4036-b431-3bdd9b13cc9f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:35.523009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccccbd0-5b97-47e1-85b2-01b42556a579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:35.523963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccccbd0-5b97-47e1-85b2-01b42556a579 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.524516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccccbd0-5b97-47e1-85b2-01b42556a579 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.524980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccccbd0-5b97-47e1-85b2-01b42556a579 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.526162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccccbd0-5b97-47e1-85b2-01b42556a579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:35.526694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccccbd0-5b97-47e1-85b2-01b42556a579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:35.528275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccccbd0-5b97-47e1-85b2-01b42556a579 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:35.532602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d85c60-94c1-4d07-a5be-bd596b06d03a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:35.538225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 56/118] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:35.543726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccccbd0-5b97-47e1-85b2-01b42556a579 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:35.573347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdd4ac-c0b1-4d85-b1ba-590906aad52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:35.573979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdd4ac-c0b1-4d85-b1ba-590906aad52c None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.574261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdd4ac-c0b1-4d85-b1ba-590906aad52c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.574564 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdd4ac-c0b1-4d85-b1ba-590906aad52c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.575247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdd4ac-c0b1-4d85-b1ba-590906aad52c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:35.576684 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdd4ac-c0b1-4d85-b1ba-590906aad52c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:35.605882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 63/119] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:44:35 2022] POST /identity/v3/projects => generated 302 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:35.608418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdd4ac-c0b1-4d85-b1ba-590906aad52c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:35.675347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccccbd0-5b97-47e1-85b2-01b42556a579 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:35.717360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450a84c-ba6e-4e07-85ce-2e870fa0bd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:35.718123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450a84c-ba6e-4e07-85ce-2e870fa0bd32 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.718463 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450a84c-ba6e-4e07-85ce-2e870fa0bd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.718760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450a84c-ba6e-4e07-85ce-2e870fa0bd32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.720192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:35.731932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:35.748434 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdd4ac-c0b1-4d85-b1ba-590906aad52c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:35.789678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593b09c-0511-4d60-be69-2bd4f2ed2f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:35.790265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b09c-0511-4d60-be69-2bd4f2ed2f9d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.792254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b09c-0511-4d60-be69-2bd4f2ed2f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.792254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b09c-0511-4d60-be69-2bd4f2ed2f9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.803440 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593b09c-0511-4d60-be69-2bd4f2ed2f9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:35.804717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450a84c-ba6e-4e07-85ce-2e870fa0bd32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:35.806219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450a84c-ba6e-4e07-85ce-2e870fa0bd32 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4593b09c-0511-4d60-be69-2bd4f2ed2f9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:35.810076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:35.810990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 64/120] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:44:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:35.811802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593b09c-0511-4d60-be69-2bd4f2ed2f9d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:35.930893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 57/121] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:35.932188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dafae-56c5-4e93-804c-2f8984ccb485 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:35.932531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dafae-56c5-4e93-804c-2f8984ccb485 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:35.932863 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dafae-56c5-4e93-804c-2f8984ccb485 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:35.934296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 58/122] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:35.939196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dafae-56c5-4e93-804c-2f8984ccb485 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:35.952148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593b09c-0511-4d60-be69-2bd4f2ed2f9d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:35.997991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:36.002403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d8f05-5241-4eb8-a451-1d466a7701dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:36.010573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d8f05-5241-4eb8-a451-1d466a7701dd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.011139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d8f05-5241-4eb8-a451-1d466a7701dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.011673 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d8f05-5241-4eb8-a451-1d466a7701dd None admin] PATH_INFO: `/v3/users/74c09f97e34c428a84e13d34f5105ca9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.013686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d8f05-5241-4eb8-a451-1d466a7701dd None admin] RBAC: Authorizing `identity:get_user(user_id=74c09f97e34c428a84e13d34f5105ca9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:36.016353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d8f05-5241-4eb8-a451-1d466a7701dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:36.020236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 65/123] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:35 2022] GET /identity/v3/users/74c09f97e34c428a84e13d34f5105ca9 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:36.024380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d8f05-5241-4eb8-a451-1d466a7701dd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:36.038376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:36.088739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dafae-56c5-4e93-804c-2f8984ccb485 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:36.135284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc577e2-c119-4b4e-9952-6c56d2950217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:36.136057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc577e2-c119-4b4e-9952-6c56d2950217 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.136834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc577e2-c119-4b4e-9952-6c56d2950217 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.137214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc577e2-c119-4b4e-9952-6c56d2950217 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.138359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc577e2-c119-4b4e-9952-6c56d2950217 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:36.138776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc577e2-c119-4b4e-9952-6c56d2950217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:36.142253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc577e2-c119-4b4e-9952-6c56d2950217 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:36.144040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d8f05-5241-4eb8-a451-1d466a7701dd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:36.152784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 59/124] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:44:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:36.154172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd208e-c560-4f36-bfca-2c15f4e5d0de None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.154603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd208e-c560-4f36-bfca-2c15f4e5d0de None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.156230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd208e-c560-4f36-bfca-2c15f4e5d0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.178639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1ffd208e-c560-4f36-bfca-2c15f4e5d0de None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:36.180528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1ffd208e-c560-4f36-bfca-2c15f4e5d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:36.184905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9bb8-f222-4ab4-a8e3-baaf2fcb83a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:36.185723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9bb8-f222-4ab4-a8e3-baaf2fcb83a9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.186152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9bb8-f222-4ab4-a8e3-baaf2fcb83a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.187378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9bb8-f222-4ab4-a8e3-baaf2fcb83a9 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.189073 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9bb8-f222-4ab4-a8e3-baaf2fcb83a9 None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:36.190978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9bb8-f222-4ab4-a8e3-baaf2fcb83a9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:36.193024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 66/125] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:36 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:36.194102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9bb8-f222-4ab4-a8e3-baaf2fcb83a9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:36.274488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 60/126] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:36.274488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd208e-c560-4f36-bfca-2c15f4e5d0de None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:36.305901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:36.308539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:36.336503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9bb8-f222-4ab4-a8e3-baaf2fcb83a9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:36.373011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be239ac-ba65-4ed7-9f61-9c980a49dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:36.373965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be239ac-ba65-4ed7-9f61-9c980a49dd66 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.374418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be239ac-ba65-4ed7-9f61-9c980a49dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.374903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be239ac-ba65-4ed7-9f61-9c980a49dd66 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.376239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be239ac-ba65-4ed7-9f61-9c980a49dd66 None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:36.377874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be239ac-ba65-4ed7-9f61-9c980a49dd66 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:36.379988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 67/127] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:36 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:36.380962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be239ac-ba65-4ed7-9f61-9c980a49dd66 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:36.403991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd208e-c560-4f36-bfca-2c15f4e5d0de None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:36.448927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea459787-2255-4246-bbe0-b80ba6705ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:36.449999 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea459787-2255-4246-bbe0-b80ba6705ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.450370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea459787-2255-4246-bbe0-b80ba6705ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.450780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea459787-2255-4246-bbe0-b80ba6705ea4 None admin] PATH_INFO: `/v3/users/5716eec426c840c79a2d8f3c7ec7ad18` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.452764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea459787-2255-4246-bbe0-b80ba6705ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=5716eec426c840c79a2d8f3c7ec7ad18)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:36.454935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea459787-2255-4246-bbe0-b80ba6705ea4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:36.456661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 61/128] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:36 2022] GET /identity/v3/users/5716eec426c840c79a2d8f3c7ec7ad18 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:36.457775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea459787-2255-4246-bbe0-b80ba6705ea4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:36.495737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be239ac-ba65-4ed7-9f61-9c980a49dd66 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:36.533086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341a983-0fda-47e3-aed2-dc1b24e03eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:36.533933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341a983-0fda-47e3-aed2-dc1b24e03eaa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.534363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341a983-0fda-47e3-aed2-dc1b24e03eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.534990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341a983-0fda-47e3-aed2-dc1b24e03eaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.535928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341a983-0fda-47e3-aed2-dc1b24e03eaa None admin] RBAC: Adding query filter params (role.id=6cce74d600ba46a2b7786018d0836956, scope.project.id=395264e10f8345d89e730d4c1a103dd2, user.id=a90b5f37d8ea4b1db948a26c451368e7) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:36.536386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341a983-0fda-47e3-aed2-dc1b24e03eaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:36.537968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341a983-0fda-47e3-aed2-dc1b24e03eaa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:36.544543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 68/129] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:36 2022] GET /identity/v3/role_assignments?user.id=a90b5f37d8ea4b1db948a26c451368e7&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=6cce74d600ba46a2b7786018d0836956 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:36.545545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341a983-0fda-47e3-aed2-dc1b24e03eaa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:36.558148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:36.583703 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:36.588208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea459787-2255-4246-bbe0-b80ba6705ea4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:36.627553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92865086-408d-48d3-b320-432b0ec32f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:36.628179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92865086-408d-48d3-b320-432b0ec32f36 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.628467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92865086-408d-48d3-b320-432b0ec32f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.628744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92865086-408d-48d3-b320-432b0ec32f36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.641837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92865086-408d-48d3-b320-432b0ec32f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-92865086-408d-48d3-b320-432b0ec32f36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:36.647439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:36.648300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 62/130] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:36.649059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92865086-408d-48d3-b320-432b0ec32f36 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:36.691244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341a983-0fda-47e3-aed2-dc1b24e03eaa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:36.736956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7951e1-0ac9-4757-990d-2042e90072ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:36.737993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7951e1-0ac9-4757-990d-2042e90072ce None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.738499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7951e1-0ac9-4757-990d-2042e90072ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.738992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7951e1-0ac9-4757-990d-2042e90072ce None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2/users/74c09f97e34c428a84e13d34f5105ca9/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.741522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7951e1-0ac9-4757-990d-2042e90072ce None admin] RBAC: Authorizing `identity:create_grant(project_id=395264e10f8345d89e730d4c1a103dd2, user_id=74c09f97e34c428a84e13d34f5105ca9, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:36.743520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7951e1-0ac9-4757-990d-2042e90072ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:36.763252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 69/131] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:44:36 2022] PUT /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2/users/74c09f97e34c428a84e13d34f5105ca9/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:44:36.770639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7951e1-0ac9-4757-990d-2042e90072ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:36.842646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92865086-408d-48d3-b320-432b0ec32f36 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:36.885065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b5543-3330-4dfc-84ee-54329422c41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:36.885706 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b5543-3330-4dfc-84ee-54329422c41a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:36.885961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b5543-3330-4dfc-84ee-54329422c41a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:36.886240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b5543-3330-4dfc-84ee-54329422c41a None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:36.888739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b5543-3330-4dfc-84ee-54329422c41a None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:36.890943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b5543-3330-4dfc-84ee-54329422c41a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:36.892697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 63/132] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:36 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:36.899001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:36.899378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b5543-3330-4dfc-84ee-54329422c41a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:36.942116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:36.958485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7951e1-0ac9-4757-990d-2042e90072ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:37.037234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b5543-3330-4dfc-84ee-54329422c41a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:37.063182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4078b7-969b-44b9-b9ba-19303cd95c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:37.064187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4078b7-969b-44b9-b9ba-19303cd95c42 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.064691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4078b7-969b-44b9-b9ba-19303cd95c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.065156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4078b7-969b-44b9-b9ba-19303cd95c42 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.066078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4078b7-969b-44b9-b9ba-19303cd95c42 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:37.066602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4078b7-969b-44b9-b9ba-19303cd95c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:37.068187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4078b7-969b-44b9-b9ba-19303cd95c42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:37.078770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df83b30-ddfb-4c42-845e-f8bc25573973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:37.079459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df83b30-ddfb-4c42-845e-f8bc25573973 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.082459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df83b30-ddfb-4c42-845e-f8bc25573973 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.082459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df83b30-ddfb-4c42-845e-f8bc25573973 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2/users/5716eec426c840c79a2d8f3c7ec7ad18/roles/9ec4568f149c40c38cd07fb317a40280` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.083025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df83b30-ddfb-4c42-845e-f8bc25573973 None admin] RBAC: Authorizing `identity:create_grant(project_id=395264e10f8345d89e730d4c1a103dd2, user_id=5716eec426c840c79a2d8f3c7ec7ad18, role_id=9ec4568f149c40c38cd07fb317a40280)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:37.084797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df83b30-ddfb-4c42-845e-f8bc25573973 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:37.087210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 70/133] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:37.088682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493e9d8-db1d-4e4a-9255-2048cbe9f40d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.089187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493e9d8-db1d-4e4a-9255-2048cbe9f40d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.089801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493e9d8-db1d-4e4a-9255-2048cbe9f40d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.099140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 71/134] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:37.099140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493e9d8-db1d-4e4a-9255-2048cbe9f40d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:37.109893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 64/135] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:44:36 2022] PUT /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2/users/5716eec426c840c79a2d8f3c7ec7ad18/roles/9ec4568f149c40c38cd07fb317a40280 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:44:37.111672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79688867-676f-44c9-9a1e-b815d8d3ff97 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.113281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79688867-676f-44c9-9a1e-b815d8d3ff97 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.113920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79688867-676f-44c9-9a1e-b815d8d3ff97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.149402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-79688867-676f-44c9-9a1e-b815d8d3ff97 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:37.151207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-79688867-676f-44c9-9a1e-b815d8d3ff97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:37.240100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 65/136] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:37.264579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79688867-676f-44c9-9a1e-b815d8d3ff97 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:37.286692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:37.323160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493e9d8-db1d-4e4a-9255-2048cbe9f40d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:37.369680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:37.380468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79688867-676f-44c9-9a1e-b815d8d3ff97 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:37.381706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77864d25-b59f-4a66-9af6-bbae9907deb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:37.382376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77864d25-b59f-4a66-9af6-bbae9907deb3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.382681 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77864d25-b59f-4a66-9af6-bbae9907deb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.383025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77864d25-b59f-4a66-9af6-bbae9907deb3 None admin] PATH_INFO: `/v3/users/5716eec426c840c79a2d8f3c7ec7ad18` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.385515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77864d25-b59f-4a66-9af6-bbae9907deb3 None admin] RBAC: Authorizing `identity:get_user(user_id=5716eec426c840c79a2d8f3c7ec7ad18)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:37.387085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77864d25-b59f-4a66-9af6-bbae9907deb3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:37.388784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 72/137] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:37 2022] GET /identity/v3/users/5716eec426c840c79a2d8f3c7ec7ad18 => generated 275 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:37.394492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77864d25-b59f-4a66-9af6-bbae9907deb3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:37.423439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cf50e-d843-45c1-a837-620170570be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:37.424382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf50e-d843-45c1-a837-620170570be8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.424838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf50e-d843-45c1-a837-620170570be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.425281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf50e-d843-45c1-a837-620170570be8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.451069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cf50e-d843-45c1-a837-620170570be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d42cf50e-d843-45c1-a837-620170570be8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:37.457315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:37.458614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:37.459380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:37.470851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 66/138] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:37.495148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cf50e-d843-45c1-a837-620170570be8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:37.593277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77864d25-b59f-4a66-9af6-bbae9907deb3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:37.611360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:37.649137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cf50e-d843-45c1-a837-620170570be8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:37.669074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49a25c-0fcc-461b-9256-4fcc598adabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:37.669902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49a25c-0fcc-461b-9256-4fcc598adabd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.670289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49a25c-0fcc-461b-9256-4fcc598adabd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.670692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49a25c-0fcc-461b-9256-4fcc598adabd None admin] PATH_INFO: `/v3/projects/019c9a95e4514d839c157565d71c4500` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.675270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49a25c-0fcc-461b-9256-4fcc598adabd None admin] RBAC: Authorizing `identity:get_project(project_id=019c9a95e4514d839c157565d71c4500)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:37.677273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49a25c-0fcc-461b-9256-4fcc598adabd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:37.679547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 73/139] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:37 2022] GET /identity/v3/projects/019c9a95e4514d839c157565d71c4500 => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:37.685082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49a25c-0fcc-461b-9256-4fcc598adabd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:37.693790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a81c29-9f8c-4647-81b2-a8004b35021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:37.694708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a81c29-9f8c-4647-81b2-a8004b35021b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.695213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a81c29-9f8c-4647-81b2-a8004b35021b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.695728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a81c29-9f8c-4647-81b2-a8004b35021b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.696642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a81c29-9f8c-4647-81b2-a8004b35021b None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:37.697118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a81c29-9f8c-4647-81b2-a8004b35021b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:37.698710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a81c29-9f8c-4647-81b2-a8004b35021b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:37.708174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 67/140] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:37.714131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a81c29-9f8c-4647-81b2-a8004b35021b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:37.817509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:37.896021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a81c29-9f8c-4647-81b2-a8004b35021b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:37.900599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49a25c-0fcc-461b-9256-4fcc598adabd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:37.960651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7168800-dee6-417b-9ad0-0265e372db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:37.961328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7168800-dee6-417b-9ad0-0265e372db68 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.961615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7168800-dee6-417b-9ad0-0265e372db68 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.961893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7168800-dee6-417b-9ad0-0265e372db68 None admin] PATH_INFO: `/v3/users/5716eec426c840c79a2d8f3c7ec7ad18` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.963291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7168800-dee6-417b-9ad0-0265e372db68 None admin] RBAC: Authorizing `identity:get_user(user_id=5716eec426c840c79a2d8f3c7ec7ad18)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:37.964758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7168800-dee6-417b-9ad0-0265e372db68 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:37.966741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 68/141] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:37 2022] GET /identity/v3/users/5716eec426c840c79a2d8f3c7ec7ad18 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:37.972896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7168800-dee6-417b-9ad0-0265e372db68 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:37.985891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c933417-6203-4068-bad5-17f88ff389da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:37.986625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c933417-6203-4068-bad5-17f88ff389da None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:37.986910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c933417-6203-4068-bad5-17f88ff389da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:37.987193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c933417-6203-4068-bad5-17f88ff389da None admin] PATH_INFO: `/v3/projects/019c9a95e4514d839c157565d71c4500/users/5716eec426c840c79a2d8f3c7ec7ad18/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:37.988751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:37.995616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c933417-6203-4068-bad5-17f88ff389da None admin] RBAC: Authorizing `identity:create_grant(project_id=019c9a95e4514d839c157565d71c4500, user_id=5716eec426c840c79a2d8f3c7ec7ad18, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:37.998158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c933417-6203-4068-bad5-17f88ff389da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:38.034278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 74/142] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:44:37 2022] PUT /identity/v3/projects/019c9a95e4514d839c157565d71c4500/users/5716eec426c840c79a2d8f3c7ec7ad18/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:44:38.035815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cabbd6-fea0-4945-976c-ae41ae909fd8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:38.036281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cabbd6-fea0-4945-976c-ae41ae909fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:38.036705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cabbd6-fea0-4945-976c-ae41ae909fd8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:38.038169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 75/143] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:38.044734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cc22c-e4c6-40cd-a678-c2a0eac2ea10 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:38.045247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cc22c-e4c6-40cd-a678-c2a0eac2ea10 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:38.045698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cc22c-e4c6-40cd-a678-c2a0eac2ea10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:38.087747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-758cc22c-e4c6-40cd-a678-c2a0eac2ea10 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:38.088971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-758cc22c-e4c6-40cd-a678-c2a0eac2ea10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:38.139227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7168800-dee6-417b-9ad0-0265e372db68 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:38.179723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6f26d-3a86-49dd-9b96-ed5b1aefb530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:38.180632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6f26d-3a86-49dd-9b96-ed5b1aefb530 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:38.181062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6f26d-3a86-49dd-9b96-ed5b1aefb530 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:38.181484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6f26d-3a86-49dd-9b96-ed5b1aefb530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:38.256815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 76/144] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:38.268674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6f26d-3a86-49dd-9b96-ed5b1aefb530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:38.270216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6f26d-3a86-49dd-9b96-ed5b1aefb530 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:38.282941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:38.301607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cc22c-e4c6-40cd-a678-c2a0eac2ea10 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:38.412819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:38.458839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 69/145] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:44:38.464982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6f26d-3a86-49dd-9b96-ed5b1aefb530 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:38.497547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cc22c-e4c6-40cd-a678-c2a0eac2ea10 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:38.583430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92378d-de9c-4c05-b969-27fbd59f9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:38.584147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92378d-de9c-4c05-b969-27fbd59f9920 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:38.584426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92378d-de9c-4c05-b969-27fbd59f9920 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:38.584698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92378d-de9c-4c05-b969-27fbd59f9920 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:38.586218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92378d-de9c-4c05-b969-27fbd59f9920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:38.587708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92378d-de9c-4c05-b969-27fbd59f9920 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:38.590105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 77/146] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:38.595918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92378d-de9c-4c05-b969-27fbd59f9920 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:38.651967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6f26d-3a86-49dd-9b96-ed5b1aefb530 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:38.672935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:38.721161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1af88-f191-4742-a055-7dcec9b34ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:38.721807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1af88-f191-4742-a055-7dcec9b34ead None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:38.722085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1af88-f191-4742-a055-7dcec9b34ead None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:38.722360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1af88-f191-4742-a055-7dcec9b34ead None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:38.723673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1af88-f191-4742-a055-7dcec9b34ead None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:38.725107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1af88-f191-4742-a055-7dcec9b34ead None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:38.728072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 70/147] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:38 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:38.735105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1af88-f191-4742-a055-7dcec9b34ead None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:38.780160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92378d-de9c-4c05-b969-27fbd59f9920 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:38.835774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:38.855472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7cf20-987e-4ff8-98cb-64e2ce1ac191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:38.856100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7cf20-987e-4ff8-98cb-64e2ce1ac191 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:38.856378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7cf20-987e-4ff8-98cb-64e2ce1ac191 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:38.856651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7cf20-987e-4ff8-98cb-64e2ce1ac191 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:38.857337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7cf20-987e-4ff8-98cb-64e2ce1ac191 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:38.861565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7cf20-987e-4ff8-98cb-64e2ce1ac191 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:38.909024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 78/148] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:44:38 2022] POST /identity/v3/users => generated 283 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:38.920037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1af88-f191-4742-a055-7dcec9b34ead None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:38.959949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6265997-9b3a-4683-a4f9-db86b1530853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:38.960580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6265997-9b3a-4683-a4f9-db86b1530853 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:38.960859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6265997-9b3a-4683-a4f9-db86b1530853 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:38.961129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6265997-9b3a-4683-a4f9-db86b1530853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:38.961941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6265997-9b3a-4683-a4f9-db86b1530853 None admin] RBAC: Adding query filter params (role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=395264e10f8345d89e730d4c1a103dd2, user.id=5716eec426c840c79a2d8f3c7ec7ad18) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:38.962249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6265997-9b3a-4683-a4f9-db86b1530853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:38.963776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6265997-9b3a-4683-a4f9-db86b1530853 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:38.971856 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 71/149] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:38 2022] GET /identity/v3/role_assignments?user.id=5716eec426c840c79a2d8f3c7ec7ad18&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:39.454425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99db19e-d462-4a09-8042-d057c6660f1e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:39.454737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99db19e-d462-4a09-8042-d057c6660f1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:39.455025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99db19e-d462-4a09-8042-d057c6660f1e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:39.456405 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 79/150] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:39.472489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ef35e-c561-4688-8c6f-75d6d4ab6a39 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:39.472997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ef35e-c561-4688-8c6f-75d6d4ab6a39 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:39.473079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ef35e-c561-4688-8c6f-75d6d4ab6a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:39.492656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fb8ef35e-c561-4688-8c6f-75d6d4ab6a39 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:39.493883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fb8ef35e-c561-4688-8c6f-75d6d4ab6a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:39.538903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:39.576998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 72/151] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:39.606066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99db19e-d462-4a09-8042-d057c6660f1e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:39.738748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99db19e-d462-4a09-8042-d057c6660f1e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:39.778805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40e02-b377-4b3c-9394-f84426c4893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:39.779496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40e02-b377-4b3c-9394-f84426c4893f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:39.779739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40e02-b377-4b3c-9394-f84426c4893f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:39.780010 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40e02-b377-4b3c-9394-f84426c4893f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:39.790797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093eb13-b977-490d-804c-916c3d7a0e2a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:39.791098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093eb13-b977-490d-804c-916c3d7a0e2a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:39.791350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093eb13-b977-490d-804c-916c3d7a0e2a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:39.792778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 73/152] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:39.797218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa40e02-b377-4b3c-9394-f84426c4893f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:39.799288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c8b8-c81d-4111-a59f-72f7e47a686a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:39.799571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c8b8-c81d-4111-a59f-72f7e47a686a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:39.799843 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c8b8-c81d-4111-a59f-72f7e47a686a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3aa40e02-b377-4b3c-9394-f84426c4893f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:39.805086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:39.805992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:39.806979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 80/153] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:39.813088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40e02-b377-4b3c-9394-f84426c4893f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:39.827919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f951c8b8-c81d-4111-a59f-72f7e47a686a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:39.829570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f951c8b8-c81d-4111-a59f-72f7e47a686a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:39.869090 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:39.933955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 74/154] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:39.943048 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40e02-b377-4b3c-9394-f84426c4893f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:39.976034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c8b8-c81d-4111-a59f-72f7e47a686a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:39.984533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baafe86-5939-4268-a1e4-27ae912b7da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:39.985175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baafe86-5939-4268-a1e4-27ae912b7da1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:39.985455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baafe86-5939-4268-a1e4-27ae912b7da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:39.985731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baafe86-5939-4268-a1e4-27ae912b7da1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:39.986485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baafe86-5939-4268-a1e4-27ae912b7da1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:39.986797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baafe86-5939-4268-a1e4-27ae912b7da1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:39.988192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baafe86-5939-4268-a1e4-27ae912b7da1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:40.015463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 81/155] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:40.024328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baafe86-5939-4268-a1e4-27ae912b7da1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:40.137543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:40.142693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c8b8-c81d-4111-a59f-72f7e47a686a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:40.189954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ef1f-c744-45e9-9a8a-557afac50d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:40.190642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ef1f-c744-45e9-9a8a-557afac50d16 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:40.190924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ef1f-c744-45e9-9a8a-557afac50d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:40.191194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654ef1f-c744-45e9-9a8a-557afac50d16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:40.208267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654ef1f-c744-45e9-9a8a-557afac50d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c654ef1f-c744-45e9-9a8a-557afac50d16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:40.213242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:40.214152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:40.215511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 75/156] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:44:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:44:40.220561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ef1f-c744-45e9-9a8a-557afac50d16 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:40.244167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baafe86-5939-4268-a1e4-27ae912b7da1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:40.246056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:40.290552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f2cba-c406-4559-9045-5295a8e1ddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:40.291633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2cba-c406-4559-9045-5295a8e1ddb4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:40.292125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2cba-c406-4559-9045-5295a8e1ddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:40.292604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2cba-c406-4559-9045-5295a8e1ddb4 None admin] PATH_INFO: `/v3/users/a90b5f37d8ea4b1db948a26c451368e7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:40.294141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f2cba-c406-4559-9045-5295a8e1ddb4 None admin] RBAC: Authorizing `identity:get_user(user_id=a90b5f37d8ea4b1db948a26c451368e7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:40.295852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f2cba-c406-4559-9045-5295a8e1ddb4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:40.298157 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 82/157] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:40 2022] GET /identity/v3/users/a90b5f37d8ea4b1db948a26c451368e7 => generated 289 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:40.301972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070e3e-973e-497b-9202-53279e4c8c3b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:40.302526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070e3e-973e-497b-9202-53279e4c8c3b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:40.303286 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070e3e-973e-497b-9202-53279e4c8c3b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:40.305317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 83/158] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:40.306261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0070e3e-973e-497b-9202-53279e4c8c3b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:40.367398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654ef1f-c744-45e9-9a8a-557afac50d16 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:40.433299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1acdf0-14db-47fc-8665-b178c52c35a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:40.434466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1acdf0-14db-47fc-8665-b178c52c35a4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:40.434466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1acdf0-14db-47fc-8665-b178c52c35a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:40.434786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1acdf0-14db-47fc-8665-b178c52c35a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:40.435641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1acdf0-14db-47fc-8665-b178c52c35a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:40.435924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1acdf0-14db-47fc-8665-b178c52c35a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:40.437430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1acdf0-14db-47fc-8665-b178c52c35a4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:40.456864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 76/159] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:40.460963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496719d-5340-49d4-aa8e-247166d8235e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:40.461255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496719d-5340-49d4-aa8e-247166d8235e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:40.462890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496719d-5340-49d4-aa8e-247166d8235e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:40.487489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0070e3e-973e-497b-9202-53279e4c8c3b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:40.491169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3496719d-5340-49d4-aa8e-247166d8235e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:40.492944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3496719d-5340-49d4-aa8e-247166d8235e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:40.516845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:40.564698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fd100-17fe-47c9-ad58-59fffc57826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:40.565274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fd100-17fe-47c9-ad58-59fffc57826f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:40.565548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fd100-17fe-47c9-ad58-59fffc57826f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:40.565817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fd100-17fe-47c9-ad58-59fffc57826f None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:40.571193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fd100-17fe-47c9-ad58-59fffc57826f None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:40.572674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fd100-17fe-47c9-ad58-59fffc57826f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:40.574507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 84/160] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:40 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:40.575169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fd100-17fe-47c9-ad58-59fffc57826f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:40.584688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 77/161] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:40.586048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496719d-5340-49d4-aa8e-247166d8235e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:40.688780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:40.748118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496719d-5340-49d4-aa8e-247166d8235e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:40.786750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eaca9-69f1-499e-9256-152dca41b618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:40.787800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eaca9-69f1-499e-9256-152dca41b618 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:40.788296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eaca9-69f1-499e-9256-152dca41b618 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:40.788838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eaca9-69f1-499e-9256-152dca41b618 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2/users/a90b5f37d8ea4b1db948a26c451368e7/roles/6cce74d600ba46a2b7786018d0836956` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:40.795230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888eaca9-69f1-499e-9256-152dca41b618 None admin] RBAC: Authorizing `identity:create_grant(project_id=395264e10f8345d89e730d4c1a103dd2, user_id=a90b5f37d8ea4b1db948a26c451368e7, role_id=6cce74d600ba46a2b7786018d0836956)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:40.795771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fd100-17fe-47c9-ad58-59fffc57826f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:40.797961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888eaca9-69f1-499e-9256-152dca41b618 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:40.827686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 78/162] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:44:40 2022] PUT /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2/users/a90b5f37d8ea4b1db948a26c451368e7/roles/6cce74d600ba46a2b7786018d0836956 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:44:40.828926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eaca9-69f1-499e-9256-152dca41b618 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:40.847285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c49f34-8c7e-49ed-b91a-e7fb6c1877db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:40.848022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c49f34-8c7e-49ed-b91a-e7fb6c1877db None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:40.848317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c49f34-8c7e-49ed-b91a-e7fb6c1877db None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:40.848595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c49f34-8c7e-49ed-b91a-e7fb6c1877db None admin] PATH_INFO: `/v3/users/74c09f97e34c428a84e13d34f5105ca9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:40.857013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c49f34-8c7e-49ed-b91a-e7fb6c1877db None admin] RBAC: Authorizing `identity:get_user(user_id=74c09f97e34c428a84e13d34f5105ca9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:40.857013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c49f34-8c7e-49ed-b91a-e7fb6c1877db None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:40.857013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 85/163] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:40 2022] GET /identity/v3/users/74c09f97e34c428a84e13d34f5105ca9 => generated 247 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:40.859614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:40.860766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c49f34-8c7e-49ed-b91a-e7fb6c1877db None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:40.984816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eaca9-69f1-499e-9256-152dca41b618 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:40.990467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c49f34-8c7e-49ed-b91a-e7fb6c1877db None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:40.995104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:41.047232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d24e47-09ad-4c4a-aa3d-798f792c6d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:41.048278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d24e47-09ad-4c4a-aa3d-798f792c6d00 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:41.048779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d24e47-09ad-4c4a-aa3d-798f792c6d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:41.049250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d24e47-09ad-4c4a-aa3d-798f792c6d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:41.115315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc98a9-da1f-48cb-9d94-2dfeed06cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:41.115875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc98a9-da1f-48cb-9d94-2dfeed06cd77 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:41.116152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc98a9-da1f-48cb-9d94-2dfeed06cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:41.116422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc98a9-da1f-48cb-9d94-2dfeed06cd77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:41.140403 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc98a9-da1f-48cb-9d94-2dfeed06cd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-17fc98a9-da1f-48cb-9d94-2dfeed06cd77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:41.145381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:41.146252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:41.147065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:41.148380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 79/164] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:44:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:41.149970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fd4d7-014c-4f44-9159-61417cc0a1ac None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:41.149970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fd4d7-014c-4f44-9159-61417cc0a1ac None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:41.149970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fd4d7-014c-4f44-9159-61417cc0a1ac None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:41.151372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 80/165] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:41.156850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fd4d7-014c-4f44-9159-61417cc0a1ac None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:41.172454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d24e47-09ad-4c4a-aa3d-798f792c6d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:41.173951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d24e47-09ad-4c4a-aa3d-798f792c6d00 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:41.289721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 86/166] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:44:41.291043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ba834-de0f-4fc7-b5d0-490ed5b46b62 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:41.291335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ba834-de0f-4fc7-b5d0-490ed5b46b62 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:41.291611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ba834-de0f-4fc7-b5d0-490ed5b46b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:41.307922 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:41.318825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:41.326261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a4ba834-de0f-4fc7-b5d0-490ed5b46b62 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:41.328178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a4ba834-de0f-4fc7-b5d0-490ed5b46b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:41.374530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fd4d7-014c-4f44-9159-61417cc0a1ac None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:41.438749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018a8ef-cb36-402b-be53-8a89ee599348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:41.439797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018a8ef-cb36-402b-be53-8a89ee599348 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:41.440301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018a8ef-cb36-402b-be53-8a89ee599348 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:41.440785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018a8ef-cb36-402b-be53-8a89ee599348 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:41.441814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018a8ef-cb36-402b-be53-8a89ee599348 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:41.442369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018a8ef-cb36-402b-be53-8a89ee599348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:41.443998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018a8ef-cb36-402b-be53-8a89ee599348 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:41.452692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 87/167] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:41.453974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ba834-de0f-4fc7-b5d0-490ed5b46b62 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:41.456597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 81/168] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:44:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:41.464762 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018a8ef-cb36-402b-be53-8a89ee599348 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:41.653059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ba834-de0f-4fc7-b5d0-490ed5b46b62 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:41.657313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018a8ef-cb36-402b-be53-8a89ee599348 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:41.719119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326599a-e390-4a51-aade-6611549ae862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:41.720120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c756ad0-d2f7-4240-a4af-822d2a7da98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:41.721372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c756ad0-d2f7-4240-a4af-822d2a7da98d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:41.722284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326599a-e390-4a51-aade-6611549ae862 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:41.722830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c756ad0-d2f7-4240-a4af-822d2a7da98d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:41.723299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c756ad0-d2f7-4240-a4af-822d2a7da98d None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:41.723779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326599a-e390-4a51-aade-6611549ae862 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:41.724895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326599a-e390-4a51-aade-6611549ae862 None admin] PATH_INFO: `/v3/users/5716eec426c840c79a2d8f3c7ec7ad18` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:41.726370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c756ad0-d2f7-4240-a4af-822d2a7da98d None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:41.727319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326599a-e390-4a51-aade-6611549ae862 None admin] RBAC: Authorizing `identity:get_user(user_id=5716eec426c840c79a2d8f3c7ec7ad18)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:41.728975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c756ad0-d2f7-4240-a4af-822d2a7da98d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:41.730640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326599a-e390-4a51-aade-6611549ae862 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:41.732568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 88/169] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:41 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:41.735103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 82/170] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:41 2022] GET /identity/v3/users/5716eec426c840c79a2d8f3c7ec7ad18 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:41.736202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c756ad0-d2f7-4240-a4af-822d2a7da98d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:41.740206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:41.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326599a-e390-4a51-aade-6611549ae862 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:41.745976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:41.919280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326599a-e390-4a51-aade-6611549ae862 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:41.925009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c756ad0-d2f7-4240-a4af-822d2a7da98d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:41.998567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762c852-fb5a-4e68-a3f8-78644870dcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:42.000526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762c852-fb5a-4e68-a3f8-78644870dcff None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:42.001333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762c852-fb5a-4e68-a3f8-78644870dcff None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:42.002253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762c852-fb5a-4e68-a3f8-78644870dcff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:42.003707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762c852-fb5a-4e68-a3f8-78644870dcff None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=395264e10f8345d89e730d4c1a103dd2, user.id=74c09f97e34c428a84e13d34f5105ca9) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:42.004630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762c852-fb5a-4e68-a3f8-78644870dcff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:42.011224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762c852-fb5a-4e68-a3f8-78644870dcff None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:42.012079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1b776-bf6f-4bf0-8410-71d6bb856ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:42.012929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1b776-bf6f-4bf0-8410-71d6bb856ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:42.015418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1b776-bf6f-4bf0-8410-71d6bb856ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:42.015898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1b776-bf6f-4bf0-8410-71d6bb856ac7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:42.031163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 83/171] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:41 2022] GET /identity/v3/role_assignments?user.id=74c09f97e34c428a84e13d34f5105ca9&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:42.032103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762c852-fb5a-4e68-a3f8-78644870dcff None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:42.047523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd1b776-bf6f-4bf0-8410-71d6bb856ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ffd1b776-bf6f-4bf0-8410-71d6bb856ac7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:42.054362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:42.055253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:42.056142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:42.057297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 89/172] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:42.063568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1b776-bf6f-4bf0-8410-71d6bb856ac7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:42.196912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:42.210819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:42.255453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1b776-bf6f-4bf0-8410-71d6bb856ac7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:42.272024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762c852-fb5a-4e68-a3f8-78644870dcff None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:42.302341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ae6cd-0fba-4d0f-a232-dce4d5e9924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:42.303112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ae6cd-0fba-4d0f-a232-dce4d5e9924b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:42.303483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ae6cd-0fba-4d0f-a232-dce4d5e9924b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:42.304247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ae6cd-0fba-4d0f-a232-dce4d5e9924b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:42.305068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ae6cd-0fba-4d0f-a232-dce4d5e9924b None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:42.305446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ae6cd-0fba-4d0f-a232-dce4d5e9924b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:42.306944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ae6cd-0fba-4d0f-a232-dce4d5e9924b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:42.332446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 90/173] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:42.337927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ae6cd-0fba-4d0f-a232-dce4d5e9924b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:42.345299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4604f-0cfb-44e4-89a1-55ec75262681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:42.345931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4604f-0cfb-44e4-89a1-55ec75262681 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:42.346209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4604f-0cfb-44e4-89a1-55ec75262681 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:42.349197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4604f-0cfb-44e4-89a1-55ec75262681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:42.505988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4604f-0cfb-44e4-89a1-55ec75262681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:42.507512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4604f-0cfb-44e4-89a1-55ec75262681 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:42.531185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ae6cd-0fba-4d0f-a232-dce4d5e9924b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:42.579175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:42.596107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1560d5-133a-44dc-b21b-fc3106d30ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:42.596769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1560d5-133a-44dc-b21b-fc3106d30ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:42.597055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1560d5-133a-44dc-b21b-fc3106d30ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:42.597334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1560d5-133a-44dc-b21b-fc3106d30ff4 None admin] PATH_INFO: `/v3/users/5716eec426c840c79a2d8f3c7ec7ad18` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:42.601365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1560d5-133a-44dc-b21b-fc3106d30ff4 None admin] RBAC: Authorizing `identity:get_user(user_id=5716eec426c840c79a2d8f3c7ec7ad18)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:42.602866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1560d5-133a-44dc-b21b-fc3106d30ff4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:42.608829 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 91/174] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:42 2022] GET /identity/v3/users/5716eec426c840c79a2d8f3c7ec7ad18 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:42.614301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1560d5-133a-44dc-b21b-fc3106d30ff4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:42.632968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:42.706719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 84/175] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:42.712499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4604f-0cfb-44e4-89a1-55ec75262681 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:42.820936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4604f-0cfb-44e4-89a1-55ec75262681 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:42.848267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1560d5-133a-44dc-b21b-fc3106d30ff4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:42.863164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51930b-0230-4447-86d1-a644be20a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:42.863827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51930b-0230-4447-86d1-a644be20a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:42.864105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51930b-0230-4447-86d1-a644be20a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:42.864379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51930b-0230-4447-86d1-a644be20a5c3 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:42.866855 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51930b-0230-4447-86d1-a644be20a5c3 None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:42.868315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51930b-0230-4447-86d1-a644be20a5c3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:42.872314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 85/176] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:42 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:42.877534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51930b-0230-4447-86d1-a644be20a5c3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:42.929477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa360d8-86d8-45d8-adda-3717d8729f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:42.930587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa360d8-86d8-45d8-adda-3717d8729f2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:42.931027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa360d8-86d8-45d8-adda-3717d8729f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:42.931319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa360d8-86d8-45d8-adda-3717d8729f2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:42.953125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:43.054194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:43.065003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa360d8-86d8-45d8-adda-3717d8729f2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:43.066825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa360d8-86d8-45d8-adda-3717d8729f2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:43.110164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51930b-0230-4447-86d1-a644be20a5c3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:43.158822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2646e4-e4c9-4502-ac38-8441cbdfbe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:43.159714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2646e4-e4c9-4502-ac38-8441cbdfbe0f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.160002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2646e4-e4c9-4502-ac38-8441cbdfbe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.160279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2646e4-e4c9-4502-ac38-8441cbdfbe0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.161517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2646e4-e4c9-4502-ac38-8441cbdfbe0f None admin] RBAC: Adding query filter params (role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=395264e10f8345d89e730d4c1a103dd2, user.id=5716eec426c840c79a2d8f3c7ec7ad18) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:43.161831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2646e4-e4c9-4502-ac38-8441cbdfbe0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:43.163645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2646e4-e4c9-4502-ac38-8441cbdfbe0f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:43.175009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 86/177] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:42 2022] GET /identity/v3/role_assignments?user.id=5716eec426c840c79a2d8f3c7ec7ad18&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:43.176750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22039d6-56b6-44c9-b827-f5a1ba009f87 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.177247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22039d6-56b6-44c9-b827-f5a1ba009f87 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.179694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22039d6-56b6-44c9-b827-f5a1ba009f87 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.183912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 87/178] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:43.185567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d33e8a-9215-482e-80c2-7834df9325e4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.186057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d33e8a-9215-482e-80c2-7834df9325e4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.186642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d33e8a-9215-482e-80c2-7834df9325e4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.188376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 88/179] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:43.191459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7b22e-e5ee-4776-93e5-380f21206e2e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.192657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7b22e-e5ee-4776-93e5-380f21206e2e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.193323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7b22e-e5ee-4776-93e5-380f21206e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.229626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-85b7b22e-e5ee-4776-93e5-380f21206e2e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:43.232537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-85b7b22e-e5ee-4776-93e5-380f21206e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:43.239750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 92/180] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:44:43.242934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a368c-b055-4cc6-a551-46fb2c2e2206 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.244433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a368c-b055-4cc6-a551-46fb2c2e2206 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.244935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a368c-b055-4cc6-a551-46fb2c2e2206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.275772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a61a368c-b055-4cc6-a551-46fb2c2e2206 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:43.277255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a61a368c-b055-4cc6-a551-46fb2c2e2206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:43.364857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 89/181] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:43.366008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7b22e-e5ee-4776-93e5-380f21206e2e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:43.397194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:43.408352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 93/182] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:43.409023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a368c-b055-4cc6-a551-46fb2c2e2206 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:43.441428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:43.484585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7b22e-e5ee-4776-93e5-380f21206e2e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:43.532225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39941fd8-f637-4bb6-ae19-37f568d50d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:43.533439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39941fd8-f637-4bb6-ae19-37f568d50d03 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.534173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39941fd8-f637-4bb6-ae19-37f568d50d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.534626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39941fd8-f637-4bb6-ae19-37f568d50d03 None admin] PATH_INFO: `/v3/projects/019c9a95e4514d839c157565d71c4500` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.536400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39941fd8-f637-4bb6-ae19-37f568d50d03 None admin] RBAC: Authorizing `identity:get_project(project_id=019c9a95e4514d839c157565d71c4500)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:43.538334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39941fd8-f637-4bb6-ae19-37f568d50d03 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:43.539250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a368c-b055-4cc6-a551-46fb2c2e2206 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:43.540398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 90/183] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:43 2022] GET /identity/v3/projects/019c9a95e4514d839c157565d71c4500 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:43.541938 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39941fd8-f637-4bb6-ae19-37f568d50d03 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:43.591410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b283b11-2406-4904-94d8-666df26273c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:43.592385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b283b11-2406-4904-94d8-666df26273c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.593388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b283b11-2406-4904-94d8-666df26273c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.593927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b283b11-2406-4904-94d8-666df26273c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.613430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b283b11-2406-4904-94d8-666df26273c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7b283b11-2406-4904-94d8-666df26273c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:43.619016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:43.620066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 94/184] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:44:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:43.620870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b283b11-2406-4904-94d8-666df26273c9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:43.663657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:43.670216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39941fd8-f637-4bb6-ae19-37f568d50d03 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:43.707573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be46f8-de75-4141-ac69-2ef3726b35f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:43.708195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be46f8-de75-4141-ac69-2ef3726b35f6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.708475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be46f8-de75-4141-ac69-2ef3726b35f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.708747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be46f8-de75-4141-ac69-2ef3726b35f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.717050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:43.720564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be46f8-de75-4141-ac69-2ef3726b35f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a1be46f8-de75-4141-ac69-2ef3726b35f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:43.725661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:43.726596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 91/185] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:44:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:43.732583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5141b-3ea4-4717-94b2-c4c126aa5eb6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.733630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5141b-3ea4-4717-94b2-c4c126aa5eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.733630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5141b-3ea4-4717-94b2-c4c126aa5eb6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.733630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 92/186] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:43.733630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5141b-3ea4-4717-94b2-c4c126aa5eb6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:43.744432 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b283b11-2406-4904-94d8-666df26273c9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:43.784536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd2545-2712-496d-aefa-b5a72467c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:43.785146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd2545-2712-496d-aefa-b5a72467c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.785459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd2545-2712-496d-aefa-b5a72467c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.785733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd2545-2712-496d-aefa-b5a72467c0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.786553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd2545-2712-496d-aefa-b5a72467c0b8 None admin] RBAC: Adding query filter params (role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=019c9a95e4514d839c157565d71c4500, user.id=5716eec426c840c79a2d8f3c7ec7ad18) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:43.786918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd2545-2712-496d-aefa-b5a72467c0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:43.788570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd2545-2712-496d-aefa-b5a72467c0b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:43.794960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 95/187] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:43 2022] GET /identity/v3/role_assignments?user.id=5716eec426c840c79a2d8f3c7ec7ad18&scope.project.id=019c9a95e4514d839c157565d71c4500&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:43.807506 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd2545-2712-496d-aefa-b5a72467c0b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:43.848717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5141b-3ea4-4717-94b2-c4c126aa5eb6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:43.892659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea47991-c915-4d10-b9ad-b71eff43d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:43.893206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea47991-c915-4d10-b9ad-b71eff43d203 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.893487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea47991-c915-4d10-b9ad-b71eff43d203 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.893867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea47991-c915-4d10-b9ad-b71eff43d203 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.894668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea47991-c915-4d10-b9ad-b71eff43d203 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:43.895055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea47991-c915-4d10-b9ad-b71eff43d203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:43.896473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea47991-c915-4d10-b9ad-b71eff43d203 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:43.906634 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 93/188] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:44:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:43.907954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8f647-b7dd-4803-b5d1-df85a597c2b5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.908242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8f647-b7dd-4803-b5d1-df85a597c2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.908518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8f647-b7dd-4803-b5d1-df85a597c2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.928736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-30d8f647-b7dd-4803-b5d1-df85a597c2b5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:43.930098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-30d8f647-b7dd-4803-b5d1-df85a597c2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:43.939956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd2545-2712-496d-aefa-b5a72467c0b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:43.946005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:43.982246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e739f5f-a869-435c-9932-e9f73cbbccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:43.983069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e739f5f-a869-435c-9932-e9f73cbbccb6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:43.984039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e739f5f-a869-435c-9932-e9f73cbbccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:43.985003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e739f5f-a869-435c-9932-e9f73cbbccb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:43.986214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e739f5f-a869-435c-9932-e9f73cbbccb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:43.986679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e739f5f-a869-435c-9932-e9f73cbbccb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:43.989131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e739f5f-a869-435c-9932-e9f73cbbccb6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:44.002174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 96/189] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:44.003600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb958e9-cca5-4318-93ec-6be0add1131f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.003892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb958e9-cca5-4318-93ec-6be0add1131f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.006561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb958e9-cca5-4318-93ec-6be0add1131f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.011224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 97/190] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:44.012607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb958e9-cca5-4318-93ec-6be0add1131f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.015943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 94/191] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:44.017724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8f647-b7dd-4803-b5d1-df85a597c2b5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.026621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:44.122993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb958e9-cca5-4318-93ec-6be0add1131f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:44.128898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8f647-b7dd-4803-b5d1-df85a597c2b5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:44.157639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273beafb-6c1e-4518-8a81-2ec0a78e1470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:44.158220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273beafb-6c1e-4518-8a81-2ec0a78e1470 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.158532 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273beafb-6c1e-4518-8a81-2ec0a78e1470 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.158803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273beafb-6c1e-4518-8a81-2ec0a78e1470 None admin] PATH_INFO: `/v3/users/3055fbc292c84152adf18a4fe96cc912` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.163694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e2437-072a-466c-ace0-8770b31f0015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:44.164328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e2437-072a-466c-ace0-8770b31f0015 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.164602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e2437-072a-466c-ace0-8770b31f0015 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.164868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e2437-072a-466c-ace0-8770b31f0015 None admin] PATH_INFO: `/v3/users/3055fbc292c84152adf18a4fe96cc912` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.172407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273beafb-6c1e-4518-8a81-2ec0a78e1470 None admin] RBAC: Authorizing `identity:get_user(user_id=3055fbc292c84152adf18a4fe96cc912)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:44.173876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273beafb-6c1e-4518-8a81-2ec0a78e1470 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:44.175544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 98/192] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/users/3055fbc292c84152adf18a4fe96cc912 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:44.176682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3b0f-f92d-416d-ad86-eb6711366f18 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.176964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3b0f-f92d-416d-ad86-eb6711366f18 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.177236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3b0f-f92d-416d-ad86-eb6711366f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.182393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e2437-072a-466c-ace0-8770b31f0015 None admin] RBAC: Authorizing `identity:get_user(user_id=3055fbc292c84152adf18a4fe96cc912)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:44.183864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e2437-072a-466c-ace0-8770b31f0015 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:44.185517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 95/193] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/users/3055fbc292c84152adf18a4fe96cc912 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:44.186271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e2437-072a-466c-ace0-8770b31f0015 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.195850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cf7f3b0f-f92d-416d-ad86-eb6711366f18 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:44.196307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:44.197058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf7f3b0f-f92d-416d-ad86-eb6711366f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:44.259291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:44.265784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 99/194] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:44.266928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f3b0f-f92d-416d-ad86-eb6711366f18 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.290285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e2437-072a-466c-ace0-8770b31f0015 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:44.324030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e25a9-c84e-4cb7-bcd3-e604f5d6affd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:44.324672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e25a9-c84e-4cb7-bcd3-e604f5d6affd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.324950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e25a9-c84e-4cb7-bcd3-e604f5d6affd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.325225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e25a9-c84e-4cb7-bcd3-e604f5d6affd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.326377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e25a9-c84e-4cb7-bcd3-e604f5d6affd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:44.328488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e25a9-c84e-4cb7-bcd3-e604f5d6affd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:44.330261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 96/195] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:44.330867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e25a9-c84e-4cb7-bcd3-e604f5d6affd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.385058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f3b0f-f92d-416d-ad86-eb6711366f18 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:44.405972 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:44.420813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33e585-389f-4bfe-b6b6-41335c8d0b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:44.421939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33e585-389f-4bfe-b6b6-41335c8d0b7d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.422218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33e585-389f-4bfe-b6b6-41335c8d0b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.422506 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33e585-389f-4bfe-b6b6-41335c8d0b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.431020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e25a9-c84e-4cb7-bcd3-e604f5d6affd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:44.464507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ed08a-db2c-4f1d-8d99-61e255b899a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:44.465126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ed08a-db2c-4f1d-8d99-61e255b899a1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.465404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ed08a-db2c-4f1d-8d99-61e255b899a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.465675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ed08a-db2c-4f1d-8d99-61e255b899a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.488873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:44.492332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33e585-389f-4bfe-b6b6-41335c8d0b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:44.494418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33e585-389f-4bfe-b6b6-41335c8d0b7d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:44.533725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ed08a-db2c-4f1d-8d99-61e255b899a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:44.535335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ed08a-db2c-4f1d-8d99-61e255b899a1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:44.594938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 100/196] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:44.595528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33e585-389f-4bfe-b6b6-41335c8d0b7d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.612389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:44.638124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 97/197] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:44.638973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ed08a-db2c-4f1d-8d99-61e255b899a1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.695858 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:44.699482 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33e585-389f-4bfe-b6b6-41335c8d0b7d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:44.734394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b366e5-0894-4865-b5ed-e0641b941f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:44.735055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b366e5-0894-4865-b5ed-e0641b941f98 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.735329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b366e5-0894-4865-b5ed-e0641b941f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.735596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b366e5-0894-4865-b5ed-e0641b941f98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.737631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ed08a-db2c-4f1d-8d99-61e255b899a1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:44.745657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b366e5-0894-4865-b5ed-e0641b941f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-20b366e5-0894-4865-b5ed-e0641b941f98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:44.751149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:44.752075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 101/198] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:44.753098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b366e5-0894-4865-b5ed-e0641b941f98 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.773873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174c3fa-cd0f-4430-8481-ec0cc4dc3509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:44.774512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c3fa-cd0f-4430-8481-ec0cc4dc3509 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.774787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c3fa-cd0f-4430-8481-ec0cc4dc3509 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.775062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c3fa-cd0f-4430-8481-ec0cc4dc3509 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.775734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174c3fa-cd0f-4430-8481-ec0cc4dc3509 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:44.777424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174c3fa-cd0f-4430-8481-ec0cc4dc3509 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:44.805224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 98/199] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:44:44 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:44.806039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174c3fa-cd0f-4430-8481-ec0cc4dc3509 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.838678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:44.871766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b366e5-0894-4865-b5ed-e0641b941f98 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:44.911355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aca7e1-a9e8-45a0-9a4d-6d3f08076426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:44.912015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aca7e1-a9e8-45a0-9a4d-6d3f08076426 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.912291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aca7e1-a9e8-45a0-9a4d-6d3f08076426 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.912560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aca7e1-a9e8-45a0-9a4d-6d3f08076426 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.913765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aca7e1-a9e8-45a0-9a4d-6d3f08076426 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:44.915257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aca7e1-a9e8-45a0-9a4d-6d3f08076426 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:44.916975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 102/200] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:44.917760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aca7e1-a9e8-45a0-9a4d-6d3f08076426 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:44.929459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174c3fa-cd0f-4430-8481-ec0cc4dc3509 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:44.950897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:44.970400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c24fb0-2a07-4579-b0b7-1e5aa73f5d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:44.971101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c24fb0-2a07-4579-b0b7-1e5aa73f5d72 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:44.971377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c24fb0-2a07-4579-b0b7-1e5aa73f5d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:44.971643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c24fb0-2a07-4579-b0b7-1e5aa73f5d72 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:44.972824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c24fb0-2a07-4579-b0b7-1e5aa73f5d72 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:44.974765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c24fb0-2a07-4579-b0b7-1e5aa73f5d72 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:44.975996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 99/201] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:44.976752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c24fb0-2a07-4579-b0b7-1e5aa73f5d72 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:45.051289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aca7e1-a9e8-45a0-9a4d-6d3f08076426 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:45.106199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:45.108531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826de1d-f612-4576-add7-fc8a701f593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:45.109083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826de1d-f612-4576-add7-fc8a701f593c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:45.109386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826de1d-f612-4576-add7-fc8a701f593c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:45.109632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c24fb0-2a07-4579-b0b7-1e5aa73f5d72 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:45.109738 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826de1d-f612-4576-add7-fc8a701f593c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:45.110395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826de1d-f612-4576-add7-fc8a701f593c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:45.111588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826de1d-f612-4576-add7-fc8a701f593c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:45.113002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826de1d-f612-4576-add7-fc8a701f593c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:45.122148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 103/202] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:45.122867 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826de1d-f612-4576-add7-fc8a701f593c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:45.146794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8293c4-ab68-4fd6-9b3f-35280db7101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:45.147422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8293c4-ab68-4fd6-9b3f-35280db7101f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:45.147702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8293c4-ab68-4fd6-9b3f-35280db7101f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:45.147970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8293c4-ab68-4fd6-9b3f-35280db7101f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:45.148773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8293c4-ab68-4fd6-9b3f-35280db7101f None admin] RBAC: Adding query filter params (role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae, user.id=3055fbc292c84152adf18a4fe96cc912) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:45.149082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8293c4-ab68-4fd6-9b3f-35280db7101f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:45.150595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8293c4-ab68-4fd6-9b3f-35280db7101f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:45.156862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 100/203] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:44 2022] GET /identity/v3/role_assignments?user.id=3055fbc292c84152adf18a4fe96cc912&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:45.157996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8293c4-ab68-4fd6-9b3f-35280db7101f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:45.235283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:45.251008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826de1d-f612-4576-add7-fc8a701f593c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:45.295533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38f838-096b-42de-88fd-891d2944e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:45.296656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38f838-096b-42de-88fd-891d2944e45c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:45.297136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38f838-096b-42de-88fd-891d2944e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:45.297614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38f838-096b-42de-88fd-891d2944e45c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:45.301813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38f838-096b-42de-88fd-891d2944e45c None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae, user.id=3055fbc292c84152adf18a4fe96cc912) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:45.302475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38f838-096b-42de-88fd-891d2944e45c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:45.304512 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38f838-096b-42de-88fd-891d2944e45c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:45.311183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 104/204] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:45 2022] GET /identity/v3/role_assignments?user.id=3055fbc292c84152adf18a4fe96cc912&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:45.322353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8293c4-ab68-4fd6-9b3f-35280db7101f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:45.365672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bc0ad-2b77-412e-a5b3-78346bf2fc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:45.366336 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bc0ad-2b77-412e-a5b3-78346bf2fc96 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:45.366660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bc0ad-2b77-412e-a5b3-78346bf2fc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:45.366938 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bc0ad-2b77-412e-a5b3-78346bf2fc96 None admin] PATH_INFO: `/v3/users/a90b5f37d8ea4b1db948a26c451368e7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:45.368162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bc0ad-2b77-412e-a5b3-78346bf2fc96 None admin] RBAC: Authorizing `identity:get_user(user_id=a90b5f37d8ea4b1db948a26c451368e7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:45.369636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bc0ad-2b77-412e-a5b3-78346bf2fc96 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:45.371551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 101/205] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:45 2022] GET /identity/v3/users/a90b5f37d8ea4b1db948a26c451368e7 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:45.377533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38f838-096b-42de-88fd-891d2944e45c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:45.511924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38f838-096b-42de-88fd-891d2944e45c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:45.550381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf159-e933-462f-a0ee-91a4e5bfcd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:45.551043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf159-e933-462f-a0ee-91a4e5bfcd27 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:45.551325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf159-e933-462f-a0ee-91a4e5bfcd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:45.551596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf159-e933-462f-a0ee-91a4e5bfcd27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:45.574672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:45.624039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaf159-e933-462f-a0ee-91a4e5bfcd27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:45.626200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaf159-e933-462f-a0ee-91a4e5bfcd27 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:45.733271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 105/206] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:45.740027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bc0ad-2b77-412e-a5b3-78346bf2fc96 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:45.780822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:45.877509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bc0ad-2b77-412e-a5b3-78346bf2fc96 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:45.917423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ca9b8-31db-418f-a837-a0e65932061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:45.918418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ca9b8-31db-418f-a837-a0e65932061e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:45.918955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ca9b8-31db-418f-a837-a0e65932061e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:45.919435 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ca9b8-31db-418f-a837-a0e65932061e None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:45.920818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ca9b8-31db-418f-a837-a0e65932061e None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:45.923363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ca9b8-31db-418f-a837-a0e65932061e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:45.925608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 102/207] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:45 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:45.931039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf159-e933-462f-a0ee-91a4e5bfcd27 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:46.002460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:46.062818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf159-e933-462f-a0ee-91a4e5bfcd27 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:46.098200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79624a8f-95d1-4698-bf9a-a85f21380810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:46.099168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79624a8f-95d1-4698-bf9a-a85f21380810 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:46.099664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79624a8f-95d1-4698-bf9a-a85f21380810 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:46.100128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79624a8f-95d1-4698-bf9a-a85f21380810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:46.101099 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79624a8f-95d1-4698-bf9a-a85f21380810 None admin] RBAC: Adding query filter params (role.id=6cce74d600ba46a2b7786018d0836956, scope.project.id=395264e10f8345d89e730d4c1a103dd2, user.id=a90b5f37d8ea4b1db948a26c451368e7) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:46.101619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79624a8f-95d1-4698-bf9a-a85f21380810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:46.103273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79624a8f-95d1-4698-bf9a-a85f21380810 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:46.110508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 106/208] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:45 2022] GET /identity/v3/role_assignments?user.id=a90b5f37d8ea4b1db948a26c451368e7&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=6cce74d600ba46a2b7786018d0836956 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:47.253695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36782418-6933-4a51-8967-abda1ddf89b6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.254225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36782418-6933-4a51-8967-abda1ddf89b6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.254225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36782418-6933-4a51-8967-abda1ddf89b6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.255725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 103/209] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:47.261154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebabc1c-e104-4fb9-b5c4-b85e5ef22a71 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.261547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebabc1c-e104-4fb9-b5c4-b85e5ef22a71 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.261712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebabc1c-e104-4fb9-b5c4-b85e5ef22a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.278871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ed39-3cf4-41ea-986b-75f43cfd8c7f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.279117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ed39-3cf4-41ea-986b-75f43cfd8c7f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.279500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ed39-3cf4-41ea-986b-75f43cfd8c7f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.281140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6ebabc1c-e104-4fb9-b5c4-b85e5ef22a71 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:47.281685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 104/210] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:47.282411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ebabc1c-e104-4fb9-b5c4-b85e5ef22a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:47.286903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6a170-d88f-4c06-914c-759ff2256764 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.287265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6a170-d88f-4c06-914c-759ff2256764 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.287622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6a170-d88f-4c06-914c-759ff2256764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.306942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f3d6a170-d88f-4c06-914c-759ff2256764 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:47.308363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f3d6a170-d88f-4c06-914c-759ff2256764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:47.354279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 107/211] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:47.377852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebabc1c-e104-4fb9-b5c4-b85e5ef22a71 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:47.390657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 105/212] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:47.411785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6a170-d88f-4c06-914c-759ff2256764 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:47.427316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:47.462873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:47.481981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebabc1c-e104-4fb9-b5c4-b85e5ef22a71 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:47.522955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26179878-efbe-45ef-a55a-e1f66459ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:47.524129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26179878-efbe-45ef-a55a-e1f66459ca1f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.524929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26179878-efbe-45ef-a55a-e1f66459ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.525545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26179878-efbe-45ef-a55a-e1f66459ca1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.534368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6a170-d88f-4c06-914c-759ff2256764 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:47.541308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26179878-efbe-45ef-a55a-e1f66459ca1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-26179878-efbe-45ef-a55a-e1f66459ca1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:47.548324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:47.549412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:47.550415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:47.553090 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 108/213] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:47.559171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26179878-efbe-45ef-a55a-e1f66459ca1f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:47.579095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7291d-0d0c-4ed8-8a6f-7518545a3db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:47.579911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7291d-0d0c-4ed8-8a6f-7518545a3db6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.580281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7291d-0d0c-4ed8-8a6f-7518545a3db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.580800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7291d-0d0c-4ed8-8a6f-7518545a3db6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.582469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7291d-0d0c-4ed8-8a6f-7518545a3db6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:47.584307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7291d-0d0c-4ed8-8a6f-7518545a3db6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:47.586679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 106/214] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:47.591880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7291d-0d0c-4ed8-8a6f-7518545a3db6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:47.682388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26179878-efbe-45ef-a55a-e1f66459ca1f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:47.690649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:47.690784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:47.696900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7291d-0d0c-4ed8-8a6f-7518545a3db6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:47.717663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fd9de-f001-481a-b248-11686e75b5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:47.718317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fd9de-f001-481a-b248-11686e75b5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.718619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fd9de-f001-481a-b248-11686e75b5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.718920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fd9de-f001-481a-b248-11686e75b5c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.719636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fd9de-f001-481a-b248-11686e75b5c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:47.719934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fd9de-f001-481a-b248-11686e75b5c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:47.721314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fd9de-f001-481a-b248-11686e75b5c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:47.729244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 109/215] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:47.729244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d89d2-32b9-4e72-9fb6-08adbd8272b6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.729244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d89d2-32b9-4e72-9fb6-08adbd8272b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.729244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d89d2-32b9-4e72-9fb6-08adbd8272b6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.729244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 110/216] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:47.729244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55538bd-b9cd-413c-9e25-c2da373dabe0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.729244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55538bd-b9cd-413c-9e25-c2da373dabe0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.735035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55538bd-b9cd-413c-9e25-c2da373dabe0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.735575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 111/217] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:47.736129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55538bd-b9cd-413c-9e25-c2da373dabe0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:47.740121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27dd446-fd88-4f8c-953f-9826baced040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:47.740763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dd446-fd88-4f8c-953f-9826baced040 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.741050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dd446-fd88-4f8c-953f-9826baced040 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.741326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dd446-fd88-4f8c-953f-9826baced040 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.742026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dd446-fd88-4f8c-953f-9826baced040 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:47.743518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dd446-fd88-4f8c-953f-9826baced040 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:47.766267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 107/218] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:44:47 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:47.767532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5311667-4d54-4d0f-a12d-abbb2fe5631f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.767825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5311667-4d54-4d0f-a12d-abbb2fe5631f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.768085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5311667-4d54-4d0f-a12d-abbb2fe5631f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.789036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b5311667-4d54-4d0f-a12d-abbb2fe5631f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:47.790222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b5311667-4d54-4d0f-a12d-abbb2fe5631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:47.855152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55538bd-b9cd-413c-9e25-c2da373dabe0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:47.865300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 108/219] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:47.866629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc3b05-e429-4696-aa72-f6b2340baa13 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.866917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc3b05-e429-4696-aa72-f6b2340baa13 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.867232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc3b05-e429-4696-aa72-f6b2340baa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.902290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-39dc3b05-e429-4696-aa72-f6b2340baa13 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:47.902723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dcdc3-3e38-4a92-818a-ee5c2c174ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:47.903556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-39dc3b05-e429-4696-aa72-f6b2340baa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:47.903664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dcdc3-3e38-4a92-818a-ee5c2c174ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:47.903839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dcdc3-3e38-4a92-818a-ee5c2c174ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:47.904105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dcdc3-3e38-4a92-818a-ee5c2c174ae7 None admin] PATH_INFO: `/v3/users/46298a6053564039a703effda1c0020d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:47.924331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dcdc3-3e38-4a92-818a-ee5c2c174ae7 None admin] RBAC: Authorizing `identity:get_user(user_id=46298a6053564039a703effda1c0020d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:47.926254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dcdc3-3e38-4a92-818a-ee5c2c174ae7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:47.927726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 112/220] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:47 2022] GET /identity/v3/users/46298a6053564039a703effda1c0020d => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:47.928169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dcdc3-3e38-4a92-818a-ee5c2c174ae7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:47.961000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:47.982415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 109/221] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:47.983979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc3b05-e429-4696-aa72-f6b2340baa13 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:47.999228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:48.054798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dcdc3-3e38-4a92-818a-ee5c2c174ae7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:48.097761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48a03f-e27b-4f76-9c08-95dd413eb980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:48.098416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48a03f-e27b-4f76-9c08-95dd413eb980 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:48.098742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48a03f-e27b-4f76-9c08-95dd413eb980 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:48.099018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48a03f-e27b-4f76-9c08-95dd413eb980 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:48.109826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a48a03f-e27b-4f76-9c08-95dd413eb980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:48.110020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc3b05-e429-4696-aa72-f6b2340baa13 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2a48a03f-e27b-4f76-9c08-95dd413eb980 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:48.114846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:48.115740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:48.116714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 113/222] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:44:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:48.117461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48a03f-e27b-4f76-9c08-95dd413eb980 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:48.152343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db8a70-159c-481a-846a-c7fb4c9c4b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:48.153256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db8a70-159c-481a-846a-c7fb4c9c4b8f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:48.154006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db8a70-159c-481a-846a-c7fb4c9c4b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:48.154283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db8a70-159c-481a-846a-c7fb4c9c4b8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:48.232498 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:48.247523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db8a70-159c-481a-846a-c7fb4c9c4b8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:48.249960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db8a70-159c-481a-846a-c7fb4c9c4b8f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:48.251533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48a03f-e27b-4f76-9c08-95dd413eb980 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:48.267709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:48.297780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f2a46-a024-4116-89ee-15820500d0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:48.298467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f2a46-a024-4116-89ee-15820500d0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:48.298748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f2a46-a024-4116-89ee-15820500d0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:48.299023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f2a46-a024-4116-89ee-15820500d0a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:48.313745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f2a46-a024-4116-89ee-15820500d0a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-502f2a46-a024-4116-89ee-15820500d0a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:48.320655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:48.321557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:48.322779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:48.326703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 114/223] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:44:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:48.328229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f2a46-a024-4116-89ee-15820500d0a7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:48.381101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 110/224] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:48.382381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db8a70-159c-481a-846a-c7fb4c9c4b8f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:48.453138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f2a46-a024-4116-89ee-15820500d0a7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:48.490623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b5d56-9b04-4a12-a2f1-8d49b677dd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:48.491346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b5d56-9b04-4a12-a2f1-8d49b677dd9f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:48.491626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b5d56-9b04-4a12-a2f1-8d49b677dd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:48.491899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b5d56-9b04-4a12-a2f1-8d49b677dd9f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:48.492715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b5d56-9b04-4a12-a2f1-8d49b677dd9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:48.493016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b5d56-9b04-4a12-a2f1-8d49b677dd9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:48.494623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b5d56-9b04-4a12-a2f1-8d49b677dd9f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:48.509184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 115/225] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:48.510267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b5d56-9b04-4a12-a2f1-8d49b677dd9f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:48.512312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db8a70-159c-481a-846a-c7fb4c9c4b8f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:48.522902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:48.537292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:48.566464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175a63-41c5-4c37-b16c-3ade58006cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:48.567250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175a63-41c5-4c37-b16c-3ade58006cff None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:48.567539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175a63-41c5-4c37-b16c-3ade58006cff None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:48.567810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175a63-41c5-4c37-b16c-3ade58006cff None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:48.568924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9175a63-41c5-4c37-b16c-3ade58006cff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:48.569223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9175a63-41c5-4c37-b16c-3ade58006cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:48.570637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9175a63-41c5-4c37-b16c-3ade58006cff None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:48.582862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 111/226] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:44:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:48.583705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175a63-41c5-4c37-b16c-3ade58006cff None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:48.626861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b5d56-9b04-4a12-a2f1-8d49b677dd9f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:48.664163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd577c2c-c7a5-4a74-9df1-f3c7cacda92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:48.664935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd577c2c-c7a5-4a74-9df1-f3c7cacda92a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:48.665276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd577c2c-c7a5-4a74-9df1-f3c7cacda92a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:48.665623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd577c2c-c7a5-4a74-9df1-f3c7cacda92a None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:48.667094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd577c2c-c7a5-4a74-9df1-f3c7cacda92a None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:48.668702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd577c2c-c7a5-4a74-9df1-f3c7cacda92a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:48.670586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 116/227] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:48 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:48.676848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd577c2c-c7a5-4a74-9df1-f3c7cacda92a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:48.713699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175a63-41c5-4c37-b16c-3ade58006cff None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:48.752062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e9b53-2a6a-4aba-b9db-0666ba91239c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:48.752770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e9b53-2a6a-4aba-b9db-0666ba91239c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:48.753045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e9b53-2a6a-4aba-b9db-0666ba91239c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:48.753318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e9b53-2a6a-4aba-b9db-0666ba91239c None admin] PATH_INFO: `/v3/users/3055fbc292c84152adf18a4fe96cc912` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:48.754526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e9b53-2a6a-4aba-b9db-0666ba91239c None admin] RBAC: Authorizing `identity:get_user(user_id=3055fbc292c84152adf18a4fe96cc912)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:48.756200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e9b53-2a6a-4aba-b9db-0666ba91239c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:48.758092 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 112/228] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:48 2022] GET /identity/v3/users/3055fbc292c84152adf18a4fe96cc912 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:48.758876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e9b53-2a6a-4aba-b9db-0666ba91239c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:48.768877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:48.796311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd577c2c-c7a5-4a74-9df1-f3c7cacda92a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:48.804904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:48.835832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7e350-f303-416c-9fb3-539c9ff91bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:48.836406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7e350-f303-416c-9fb3-539c9ff91bcc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:48.836685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7e350-f303-416c-9fb3-539c9ff91bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:48.836959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7e350-f303-416c-9fb3-539c9ff91bcc None admin] PATH_INFO: `/v3/users/3055fbc292c84152adf18a4fe96cc912` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:48.838205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7e350-f303-416c-9fb3-539c9ff91bcc None admin] RBAC: Authorizing `identity:get_user(user_id=3055fbc292c84152adf18a4fe96cc912)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:48.839865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7e350-f303-416c-9fb3-539c9ff91bcc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:48.841879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 117/229] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:48 2022] GET /identity/v3/users/3055fbc292c84152adf18a4fe96cc912 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:48.842894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7e350-f303-416c-9fb3-539c9ff91bcc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:48.864983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e9b53-2a6a-4aba-b9db-0666ba91239c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:48.901046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb8bdb-dc53-454b-9de1-760bc7aff53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:48.902156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb8bdb-dc53-454b-9de1-760bc7aff53f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:48.902463 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb8bdb-dc53-454b-9de1-760bc7aff53f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:48.902745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb8bdb-dc53-454b-9de1-760bc7aff53f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:48.903902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb8bdb-dc53-454b-9de1-760bc7aff53f None admin] RBAC: Adding query filter params (role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae, user.id=46298a6053564039a703effda1c0020d) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:48.904207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb8bdb-dc53-454b-9de1-760bc7aff53f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:48.905704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb8bdb-dc53-454b-9de1-760bc7aff53f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:48.913645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 113/230] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:48 2022] GET /identity/v3/role_assignments?user.id=46298a6053564039a703effda1c0020d&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:48.914275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb8bdb-dc53-454b-9de1-760bc7aff53f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:49.002418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7e350-f303-416c-9fb3-539c9ff91bcc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:49.062494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2e986-f75d-46ec-b638-0b2221de01c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:49.062494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2e986-f75d-46ec-b638-0b2221de01c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:49.062494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2e986-f75d-46ec-b638-0b2221de01c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:49.062494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2e986-f75d-46ec-b638-0b2221de01c0 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:49.063137 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2e986-f75d-46ec-b638-0b2221de01c0 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:49.063137 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2e986-f75d-46ec-b638-0b2221de01c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:49.063793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 118/231] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:48 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:49.069339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2e986-f75d-46ec-b638-0b2221de01c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:49.082464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb8bdb-dc53-454b-9de1-760bc7aff53f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:49.086665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:49.099726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:49.137123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d5d43-3f3f-4cae-bd72-4d736bd7980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:49.138132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d5d43-3f3f-4cae-bd72-4d736bd7980a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:49.138652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d5d43-3f3f-4cae-bd72-4d736bd7980a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:49.140949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d5d43-3f3f-4cae-bd72-4d736bd7980a None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:49.143830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d5d43-3f3f-4cae-bd72-4d736bd7980a None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:49.145625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d5d43-3f3f-4cae-bd72-4d736bd7980a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:49.147722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 114/232] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:48 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:49.154170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d5d43-3f3f-4cae-bd72-4d736bd7980a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:49.205201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2e986-f75d-46ec-b638-0b2221de01c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:49.264104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d5d43-3f3f-4cae-bd72-4d736bd7980a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:49.279246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c4fc9-6deb-45bf-aaec-07793dbcc110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:49.280009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4fc9-6deb-45bf-aaec-07793dbcc110 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:49.280943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4fc9-6deb-45bf-aaec-07793dbcc110 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:49.281230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4fc9-6deb-45bf-aaec-07793dbcc110 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/users/3055fbc292c84152adf18a4fe96cc912/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:49.283593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c4fc9-6deb-45bf-aaec-07793dbcc110 None admin] RBAC: Authorizing `identity:create_grant(project_id=733b6ca2bdf14132a5de09c5371d80ae, user_id=3055fbc292c84152adf18a4fe96cc912, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:49.284898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c4fc9-6deb-45bf-aaec-07793dbcc110 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:49.302169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56a8a0-c551-448c-a3ff-58f6ed52cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:49.303541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a8a0-c551-448c-a3ff-58f6ed52cb85 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:49.304054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a8a0-c551-448c-a3ff-58f6ed52cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:49.304545 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a8a0-c551-448c-a3ff-58f6ed52cb85 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/users/3055fbc292c84152adf18a4fe96cc912/roles/9ec4568f149c40c38cd07fb317a40280` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:49.307315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56a8a0-c551-448c-a3ff-58f6ed52cb85 None admin] RBAC: Authorizing `identity:create_grant(project_id=733b6ca2bdf14132a5de09c5371d80ae, user_id=3055fbc292c84152adf18a4fe96cc912, role_id=9ec4568f149c40c38cd07fb317a40280)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:49.308969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56a8a0-c551-448c-a3ff-58f6ed52cb85 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:49.309239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 119/233] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:44:49 2022] PUT /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/users/3055fbc292c84152adf18a4fe96cc912/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:44:49.329109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 115/234] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:44:49 2022] PUT /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/users/3055fbc292c84152adf18a4fe96cc912/roles/9ec4568f149c40c38cd07fb317a40280 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:44:50.154927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2897cd-1812-4886-bd4a-e5a3e30df781 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:50.156067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2897cd-1812-4886-bd4a-e5a3e30df781 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:50.156531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2897cd-1812-4886-bd4a-e5a3e30df781 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:50.158050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 120/235] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:50.174070 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a37643-81d7-4a7f-bbf5-709c755c84c3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:50.174362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a37643-81d7-4a7f-bbf5-709c755c84c3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:50.174675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a37643-81d7-4a7f-bbf5-709c755c84c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:50.203970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a0a37643-81d7-4a7f-bbf5-709c755c84c3 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:50.205390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a0a37643-81d7-4a7f-bbf5-709c755c84c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:50.244989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:50.281200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab908607-b0fc-4bbd-b1f7-05ed9bd94c18 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:50.281889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab908607-b0fc-4bbd-b1f7-05ed9bd94c18 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:50.282497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab908607-b0fc-4bbd-b1f7-05ed9bd94c18 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:50.284058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 121/236] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:44:50.289141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94bb1c-4706-41a9-a47b-02b497a3155b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:50.289645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94bb1c-4706-41a9-a47b-02b497a3155b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:50.290114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94bb1c-4706-41a9-a47b-02b497a3155b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:50.294878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 116/237] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:50.309430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5f94bb1c-4706-41a9-a47b-02b497a3155b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:50.311279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f94bb1c-4706-41a9-a47b-02b497a3155b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:50.333205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a37643-81d7-4a7f-bbf5-709c755c84c3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:50.383786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 122/238] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:50.409788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94bb1c-4706-41a9-a47b-02b497a3155b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:50.414033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:50.479122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a37643-81d7-4a7f-bbf5-709c755c84c3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:50.550055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af95193-17a1-4afe-a2aa-6071e6ce4f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:50.550654 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af95193-17a1-4afe-a2aa-6071e6ce4f9b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:50.550939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af95193-17a1-4afe-a2aa-6071e6ce4f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:50.551272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af95193-17a1-4afe-a2aa-6071e6ce4f9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:50.552552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94bb1c-4706-41a9-a47b-02b497a3155b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:50.566947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af95193-17a1-4afe-a2aa-6071e6ce4f9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8af95193-17a1-4afe-a2aa-6071e6ce4f9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:50.575554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:50.576466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:50.577959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 117/239] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:44:50.584484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af95193-17a1-4afe-a2aa-6071e6ce4f9b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:50.606807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1544a2a-aabf-4177-9075-408474e26944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:50.607743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1544a2a-aabf-4177-9075-408474e26944 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:50.608253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1544a2a-aabf-4177-9075-408474e26944 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:50.608762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1544a2a-aabf-4177-9075-408474e26944 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:50.610178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1544a2a-aabf-4177-9075-408474e26944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:50.611839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1544a2a-aabf-4177-9075-408474e26944 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:50.613674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 123/240] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:50.619224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1544a2a-aabf-4177-9075-408474e26944 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:50.627719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:50.699852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:50.725687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1544a2a-aabf-4177-9075-408474e26944 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:50.765123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48669a9e-d935-499d-9d26-a083b2e3bc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:50.766248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48669a9e-d935-499d-9d26-a083b2e3bc13 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:50.767772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48669a9e-d935-499d-9d26-a083b2e3bc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:50.768367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48669a9e-d935-499d-9d26-a083b2e3bc13 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:50.769967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48669a9e-d935-499d-9d26-a083b2e3bc13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:50.772265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48669a9e-d935-499d-9d26-a083b2e3bc13 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:50.792717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af95193-17a1-4afe-a2aa-6071e6ce4f9b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:50.799353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 124/241] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:44:50 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:50.863308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2ff29-8ac9-49cd-b7ee-5a79add1f227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:50.863879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2ff29-8ac9-49cd-b7ee-5a79add1f227 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:50.864160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2ff29-8ac9-49cd-b7ee-5a79add1f227 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:50.864438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2ff29-8ac9-49cd-b7ee-5a79add1f227 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:50.865182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2ff29-8ac9-49cd-b7ee-5a79add1f227 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:50.865486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2ff29-8ac9-49cd-b7ee-5a79add1f227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:50.866951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2ff29-8ac9-49cd-b7ee-5a79add1f227 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:50.877538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 118/242] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:50.883073 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48669a9e-d935-499d-9d26-a083b2e3bc13 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:51.025436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48669a9e-d935-499d-9d26-a083b2e3bc13 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:51.054883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:51.064626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d1ae6-4827-4838-a915-96d483988aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:51.065793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d1ae6-4827-4838-a915-96d483988aca None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:51.066391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d1ae6-4827-4838-a915-96d483988aca None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:51.066940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d1ae6-4827-4838-a915-96d483988aca None admin] PATH_INFO: `/v3/users/7b8f645f0e8449cf907423d22643a5de` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:51.082011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d1ae6-4827-4838-a915-96d483988aca None admin] RBAC: Authorizing `identity:get_user(user_id=7b8f645f0e8449cf907423d22643a5de)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:51.083817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d1ae6-4827-4838-a915-96d483988aca None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:51.085721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 125/243] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:50 2022] GET /identity/v3/users/7b8f645f0e8449cf907423d22643a5de => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:51.095549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2ff29-8ac9-49cd-b7ee-5a79add1f227 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:51.214613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:51.232074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2ff29-8ac9-49cd-b7ee-5a79add1f227 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:51.277997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186f5c4-5518-4819-bda9-8ad270bd0ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:51.278684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186f5c4-5518-4819-bda9-8ad270bd0ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:51.278968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186f5c4-5518-4819-bda9-8ad270bd0ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:51.279241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186f5c4-5518-4819-bda9-8ad270bd0ef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:51.355410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186f5c4-5518-4819-bda9-8ad270bd0ef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:51.356887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186f5c4-5518-4819-bda9-8ad270bd0ef7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:51.372245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f288d-6c5d-4016-8951-adf91b9a74cb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:51.372727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f288d-6c5d-4016-8951-adf91b9a74cb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:51.372829 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f288d-6c5d-4016-8951-adf91b9a74cb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:51.374735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 126/244] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:44:51.381247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501969-df08-41d6-860a-5df97219e6ea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:51.381795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501969-df08-41d6-860a-5df97219e6ea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:51.382338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501969-df08-41d6-860a-5df97219e6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:51.407454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-42501969-df08-41d6-860a-5df97219e6ea None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:51.409059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42501969-df08-41d6-860a-5df97219e6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:51.475175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:51.495269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 119/245] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:51.503407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186f5c4-5518-4819-bda9-8ad270bd0ef7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:51.513834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 127/246] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:51.540885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42501969-df08-41d6-860a-5df97219e6ea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:51.643230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186f5c4-5518-4819-bda9-8ad270bd0ef7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:51.667976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42501969-df08-41d6-860a-5df97219e6ea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:51.684624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:51.700883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f6d72-c83d-47e2-9285-3d54226ad3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:51.703077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f6d72-c83d-47e2-9285-3d54226ad3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:51.703605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f6d72-c83d-47e2-9285-3d54226ad3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:51.704129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f6d72-c83d-47e2-9285-3d54226ad3a1 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:51.705597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f6d72-c83d-47e2-9285-3d54226ad3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:51.707355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f6d72-c83d-47e2-9285-3d54226ad3a1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:51.709528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 120/247] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:51 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:51.718152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66036d63-a79d-4bce-98ef-325f6f46607f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:51.718152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66036d63-a79d-4bce-98ef-325f6f46607f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:51.718152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66036d63-a79d-4bce-98ef-325f6f46607f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:51.718152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66036d63-a79d-4bce-98ef-325f6f46607f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:51.719934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f6d72-c83d-47e2-9285-3d54226ad3a1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:51.728908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66036d63-a79d-4bce-98ef-325f6f46607f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-66036d63-a79d-4bce-98ef-325f6f46607f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:51.734256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:51.739371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:51.740173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 128/248] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:44:51.742358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66036d63-a79d-4bce-98ef-325f6f46607f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:51.801509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:51.859848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66036d63-a79d-4bce-98ef-325f6f46607f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:51.913281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f6d72-c83d-47e2-9285-3d54226ad3a1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:51.930715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059182f8-7e1b-4522-b946-82018dd1816f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:51.931345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059182f8-7e1b-4522-b946-82018dd1816f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:51.931624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059182f8-7e1b-4522-b946-82018dd1816f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:51.931897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059182f8-7e1b-4522-b946-82018dd1816f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:51.932619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059182f8-7e1b-4522-b946-82018dd1816f None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:51.932924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059182f8-7e1b-4522-b946-82018dd1816f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:51.934324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059182f8-7e1b-4522-b946-82018dd1816f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:51.958574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 129/249] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:51.959993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e0f2-938e-47af-a271-4662851b551e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:51.960285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e0f2-938e-47af-a271-4662851b551e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:51.960566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e0f2-938e-47af-a271-4662851b551e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:51.962016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 130/250] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:51.963376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613e0f2-938e-47af-a271-4662851b551e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:51.991607 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d10ba-5c8b-421d-9ea2-6877149f8701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:51.992160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d10ba-5c8b-421d-9ea2-6877149f8701 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:51.992444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d10ba-5c8b-421d-9ea2-6877149f8701 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:51.992718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d10ba-5c8b-421d-9ea2-6877149f8701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:51.993637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d10ba-5c8b-421d-9ea2-6877149f8701 None admin] RBAC: Adding query filter params (role.id=6cce74d600ba46a2b7786018d0836956, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae, user.id=7b8f645f0e8449cf907423d22643a5de) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:51.994054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d10ba-5c8b-421d-9ea2-6877149f8701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:51.995549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d10ba-5c8b-421d-9ea2-6877149f8701 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:52.005536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 121/251] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:51 2022] GET /identity/v3/role_assignments?user.id=7b8f645f0e8449cf907423d22643a5de&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=6cce74d600ba46a2b7786018d0836956 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:52.005794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900b4d4-8033-46be-9965-908ef8b998bf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:52.006027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900b4d4-8033-46be-9965-908ef8b998bf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:52.006307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900b4d4-8033-46be-9965-908ef8b998bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:52.030921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:52.073438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4900b4d4-8033-46be-9965-908ef8b998bf None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:52.074683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4900b4d4-8033-46be-9965-908ef8b998bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:52.175149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 122/252] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:52.176862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c426a-7d33-45ac-acf1-bbdd36e104b3 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:52.177367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c426a-7d33-45ac-acf1-bbdd36e104b3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:52.177836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c426a-7d33-45ac-acf1-bbdd36e104b3 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:52.179790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 123/253] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:52.182565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613e0f2-938e-47af-a271-4662851b551e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:52.186054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70875f69-4485-44fa-9a96-26b28d050748 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:52.186847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70875f69-4485-44fa-9a96-26b28d050748 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:52.187460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70875f69-4485-44fa-9a96-26b28d050748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:52.232378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-70875f69-4485-44fa-9a96-26b28d050748 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:52.236224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-70875f69-4485-44fa-9a96-26b28d050748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:52.264040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de1974-ed1b-441b-bf05-22cfb64130c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:52.265213 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de1974-ed1b-441b-bf05-22cfb64130c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:52.265631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de1974-ed1b-441b-bf05-22cfb64130c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:52.265914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de1974-ed1b-441b-bf05-22cfb64130c5 None admin] PATH_INFO: `/v3/users/46298a6053564039a703effda1c0020d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:52.269659 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de1974-ed1b-441b-bf05-22cfb64130c5 None admin] RBAC: Authorizing `identity:get_user(user_id=46298a6053564039a703effda1c0020d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:52.271169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de1974-ed1b-441b-bf05-22cfb64130c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:52.272942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 131/254] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:51 2022] GET /identity/v3/users/46298a6053564039a703effda1c0020d => generated 297 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:52.273813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de1974-ed1b-441b-bf05-22cfb64130c5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:52.312913 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:52.352495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 124/255] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:52.361225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70875f69-4485-44fa-9a96-26b28d050748 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:52.448844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de1974-ed1b-441b-bf05-22cfb64130c5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:52.457347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:52.487349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b7ed5-16f0-41b0-ac38-b9672c333329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:52.488541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b7ed5-16f0-41b0-ac38-b9672c333329 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:52.489043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b7ed5-16f0-41b0-ac38-b9672c333329 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:52.489526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b7ed5-16f0-41b0-ac38-b9672c333329 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:52.501769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b7ed5-16f0-41b0-ac38-b9672c333329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cf6b7ed5-16f0-41b0-ac38-b9672c333329 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:52.507197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:52.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:44:52.508934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:52.511594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 132/256] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:44:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:52.512626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b7ed5-16f0-41b0-ac38-b9672c333329 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:52.581079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70875f69-4485-44fa-9a96-26b28d050748 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:52.636440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a781ec0-0877-4535-98c6-8beb37d6cfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:52.637432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a781ec0-0877-4535-98c6-8beb37d6cfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:52.637911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a781ec0-0877-4535-98c6-8beb37d6cfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:52.638403 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a781ec0-0877-4535-98c6-8beb37d6cfd5 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:52.640428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a781ec0-0877-4535-98c6-8beb37d6cfd5 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:52.642764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a781ec0-0877-4535-98c6-8beb37d6cfd5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:52.644659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 125/257] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:52 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:52.645621 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a781ec0-0877-4535-98c6-8beb37d6cfd5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:52.708589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:52.713628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b7ed5-16f0-41b0-ac38-b9672c333329 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:52.772324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a781ec0-0877-4535-98c6-8beb37d6cfd5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:52.788171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fb114-9309-4375-8901-ea7d7b9a83f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:52.789456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fb114-9309-4375-8901-ea7d7b9a83f4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:52.790294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fb114-9309-4375-8901-ea7d7b9a83f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:52.790803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fb114-9309-4375-8901-ea7d7b9a83f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:52.806849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fb114-9309-4375-8901-ea7d7b9a83f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6d9fb114-9309-4375-8901-ea7d7b9a83f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:52.813678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:52.814620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:44:52.815417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:52.818200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 133/258] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:52.819319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fb114-9309-4375-8901-ea7d7b9a83f4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:52.844977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed4e2f-36a0-4594-aafb-91cf3091d15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:52.845531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed4e2f-36a0-4594-aafb-91cf3091d15c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:52.845806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed4e2f-36a0-4594-aafb-91cf3091d15c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:52.846074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed4e2f-36a0-4594-aafb-91cf3091d15c None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:52.846837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed4e2f-36a0-4594-aafb-91cf3091d15c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:52.847136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed4e2f-36a0-4594-aafb-91cf3091d15c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:52.849064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed4e2f-36a0-4594-aafb-91cf3091d15c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:52.868650 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:52.876970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 126/259] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:44:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:52.877904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed4e2f-36a0-4594-aafb-91cf3091d15c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:52.939369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fb114-9309-4375-8901-ea7d7b9a83f4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:52.977161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e8ab8-8b48-488d-b1e5-5c9aa7ec6ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:52.978075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e8ab8-8b48-488d-b1e5-5c9aa7ec6ba1 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:52.978489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e8ab8-8b48-488d-b1e5-5c9aa7ec6ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:52.979012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e8ab8-8b48-488d-b1e5-5c9aa7ec6ba1 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/users/46298a6053564039a703effda1c0020d/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:52.981075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e8ab8-8b48-488d-b1e5-5c9aa7ec6ba1 None admin] RBAC: Authorizing `identity:create_grant(project_id=733b6ca2bdf14132a5de09c5371d80ae, user_id=46298a6053564039a703effda1c0020d, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:52.982649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e8ab8-8b48-488d-b1e5-5c9aa7ec6ba1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:52.999698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 134/260] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:44:52 2022] PUT /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/users/46298a6053564039a703effda1c0020d/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:44:53.004593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e8ab8-8b48-488d-b1e5-5c9aa7ec6ba1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:53.189581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed4e2f-36a0-4594-aafb-91cf3091d15c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:53.202838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:53.207466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:53.215615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e8ab8-8b48-488d-b1e5-5c9aa7ec6ba1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:53.260911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a95cc-89cb-4ce4-88b6-79f3f2599407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:53.261550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a95cc-89cb-4ce4-88b6-79f3f2599407 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:53.261830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a95cc-89cb-4ce4-88b6-79f3f2599407 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:53.262102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a95cc-89cb-4ce4-88b6-79f3f2599407 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:53.262859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a95cc-89cb-4ce4-88b6-79f3f2599407 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:53.263158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a95cc-89cb-4ce4-88b6-79f3f2599407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:53.265365 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a95cc-89cb-4ce4-88b6-79f3f2599407 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:53.281883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 127/261] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:53.282094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b4361-83dc-4be7-8aee-39883ae8c1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:53.282692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b4361-83dc-4be7-8aee-39883ae8c1fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:53.282974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b4361-83dc-4be7-8aee-39883ae8c1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:53.283247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b4361-83dc-4be7-8aee-39883ae8c1fb None admin] PATH_INFO: `/v3/users/3055fbc292c84152adf18a4fe96cc912` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:53.284440 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b4361-83dc-4be7-8aee-39883ae8c1fb None admin] RBAC: Authorizing `identity:get_user(user_id=3055fbc292c84152adf18a4fe96cc912)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:53.286019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b4361-83dc-4be7-8aee-39883ae8c1fb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:53.287904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 135/262] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:52 2022] GET /identity/v3/users/3055fbc292c84152adf18a4fe96cc912 => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:53.288904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a95cc-89cb-4ce4-88b6-79f3f2599407 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:53.292994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b4361-83dc-4be7-8aee-39883ae8c1fb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:53.413067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b4361-83dc-4be7-8aee-39883ae8c1fb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:53.444920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a95cc-89cb-4ce4-88b6-79f3f2599407 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:53.453367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5dc909-e910-4b97-9e54-a8c4114e0767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:53.453990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5dc909-e910-4b97-9e54-a8c4114e0767 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:53.454281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5dc909-e910-4b97-9e54-a8c4114e0767 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:53.454592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5dc909-e910-4b97-9e54-a8c4114e0767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:53.481008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:53.516189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a347f-bae9-4e99-a2d9-8b119f15147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:53.517012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a347f-bae9-4e99-a2d9-8b119f15147f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:53.517398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a347f-bae9-4e99-a2d9-8b119f15147f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:53.517776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a347f-bae9-4e99-a2d9-8b119f15147f None admin] PATH_INFO: `/v3/users/3055fbc292c84152adf18a4fe96cc912` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:53.519175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a347f-bae9-4e99-a2d9-8b119f15147f None admin] RBAC: Authorizing `identity:get_user(user_id=3055fbc292c84152adf18a4fe96cc912)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:53.520744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a347f-bae9-4e99-a2d9-8b119f15147f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:53.522569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 128/263] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:53 2022] GET /identity/v3/users/3055fbc292c84152adf18a4fe96cc912 => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:53.528432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a347f-bae9-4e99-a2d9-8b119f15147f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:53.535679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5dc909-e910-4b97-9e54-a8c4114e0767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:53.537388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5dc909-e910-4b97-9e54-a8c4114e0767 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:53.630857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:53.645574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 136/264] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:44:53.651137 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5dc909-e910-4b97-9e54-a8c4114e0767 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:53.716375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a347f-bae9-4e99-a2d9-8b119f15147f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:53.760407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:53.776090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68353c4-777a-4aad-b943-1e30ea4c8f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:53.777042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68353c4-777a-4aad-b943-1e30ea4c8f08 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:53.777324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68353c4-777a-4aad-b943-1e30ea4c8f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:53.777594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68353c4-777a-4aad-b943-1e30ea4c8f08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:53.847058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5dc909-e910-4b97-9e54-a8c4114e0767 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:53.882183 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68353c4-777a-4aad-b943-1e30ea4c8f08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:53.883714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68353c4-777a-4aad-b943-1e30ea4c8f08 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:53.921116 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7432114-0ecf-42a5-b904-8e2fab52d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:53.921782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7432114-0ecf-42a5-b904-8e2fab52d1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:53.922066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7432114-0ecf-42a5-b904-8e2fab52d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:53.922340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7432114-0ecf-42a5-b904-8e2fab52d1d3 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:53.923580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7432114-0ecf-42a5-b904-8e2fab52d1d3 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:53.925040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7432114-0ecf-42a5-b904-8e2fab52d1d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:53.927076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 137/265] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:53 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:53.934298 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7432114-0ecf-42a5-b904-8e2fab52d1d3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:53.946834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:54.027906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 129/266] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:54.037724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68353c4-777a-4aad-b943-1e30ea4c8f08 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:54.132124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7432114-0ecf-42a5-b904-8e2fab52d1d3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:54.183163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68353c4-777a-4aad-b943-1e30ea4c8f08 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:54.190813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:54.220253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42cea1-278f-4db4-bb50-bcb54314b3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:54.220901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42cea1-278f-4db4-bb50-bcb54314b3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.221175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42cea1-278f-4db4-bb50-bcb54314b3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.221441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c42cea1-278f-4db4-bb50-bcb54314b3d0 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.222643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42cea1-278f-4db4-bb50-bcb54314b3d0 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:54.224079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c42cea1-278f-4db4-bb50-bcb54314b3d0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:54.225821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 130/267] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:54 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:54.231145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf19f5-15c4-4788-a3f2-d8618b8e0d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:54.231543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42cea1-278f-4db4-bb50-bcb54314b3d0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:54.231786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf19f5-15c4-4788-a3f2-d8618b8e0d20 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.232069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf19f5-15c4-4788-a3f2-d8618b8e0d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.232347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf19f5-15c4-4788-a3f2-d8618b8e0d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.233152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf19f5-15c4-4788-a3f2-d8618b8e0d20 None admin] RBAC: Adding query filter params (role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae, user.id=3055fbc292c84152adf18a4fe96cc912) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:54.234014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf19f5-15c4-4788-a3f2-d8618b8e0d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:54.235925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf19f5-15c4-4788-a3f2-d8618b8e0d20 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:54.240986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:54.250142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 138/268] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:53 2022] GET /identity/v3/role_assignments?user.id=3055fbc292c84152adf18a4fe96cc912&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:54.346408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c42cea1-278f-4db4-bb50-bcb54314b3d0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:54.384072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2557d-5f67-45a3-9bce-424aa84c5f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:54.384931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2557d-5f67-45a3-9bce-424aa84c5f05 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.385373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2557d-5f67-45a3-9bce-424aa84c5f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.385805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2557d-5f67-45a3-9bce-424aa84c5f05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.386843 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2557d-5f67-45a3-9bce-424aa84c5f05 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae, user.id=3055fbc292c84152adf18a4fe96cc912) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:54.387357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2557d-5f67-45a3-9bce-424aa84c5f05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:54.387760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b449f-b7a9-4d01-9c63-dd46c6dd94a1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.388047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b449f-b7a9-4d01-9c63-dd46c6dd94a1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.388328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b449f-b7a9-4d01-9c63-dd46c6dd94a1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.389440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2557d-5f67-45a3-9bce-424aa84c5f05 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:54.390267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 139/269] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:44:54.395811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948ad18-2080-479e-955e-f6eda4103594 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.396095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948ad18-2080-479e-955e-f6eda4103594 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.396375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948ad18-2080-479e-955e-f6eda4103594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.401166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 131/270] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:54 2022] GET /identity/v3/role_assignments?user.id=3055fbc292c84152adf18a4fe96cc912&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:54.439722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a948ad18-2080-479e-955e-f6eda4103594 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:54.440960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a948ad18-2080-479e-955e-f6eda4103594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:54.543638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 140/271] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:54.568490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2557d-5f67-45a3-9bce-424aa84c5f05 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:54.647556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:54.653342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040a066-f2b9-4424-897a-04d707c5edba None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.653997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040a066-f2b9-4424-897a-04d707c5edba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.654340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040a066-f2b9-4424-897a-04d707c5edba None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.655986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 141/272] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:54.662468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43045398-3368-4630-bc8d-9ad9ebe39e96 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.663004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43045398-3368-4630-bc8d-9ad9ebe39e96 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.663493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43045398-3368-4630-bc8d-9ad9ebe39e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.684219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2557d-5f67-45a3-9bce-424aa84c5f05 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:54.692731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-43045398-3368-4630-bc8d-9ad9ebe39e96 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:54.694501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-43045398-3368-4630-bc8d-9ad9ebe39e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:54.724010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9105c1-c628-45d0-a150-b3ad0e07ccd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:54.724683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9105c1-c628-45d0-a150-b3ad0e07ccd0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.724934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9105c1-c628-45d0-a150-b3ad0e07ccd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.725204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9105c1-c628-45d0-a150-b3ad0e07ccd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.740649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9105c1-c628-45d0-a150-b3ad0e07ccd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ce9105c1-c628-45d0-a150-b3ad0e07ccd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:54.748160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:54.749603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:54.750407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:54.751578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 132/273] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:54.752834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f99c66-8ad5-404c-82ac-f6952e5e6cc4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.753109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f99c66-8ad5-404c-82ac-f6952e5e6cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.753374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f99c66-8ad5-404c-82ac-f6952e5e6cc4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.754971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 133/274] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:54.756184 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f99c66-8ad5-404c-82ac-f6952e5e6cc4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:54.781886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 142/275] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:54.792154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5fc2-1179-4030-9ab3-c5e476b0135b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.792685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5fc2-1179-4030-9ab3-c5e476b0135b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.792965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa5fc2-1179-4030-9ab3-c5e476b0135b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.815049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-68aa5fc2-1179-4030-9ab3-c5e476b0135b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:54.816237 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-68aa5fc2-1179-4030-9ab3-c5e476b0135b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:54.860672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:54.865909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f99c66-8ad5-404c-82ac-f6952e5e6cc4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:54.890652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 143/276] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:54.891564 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa5fc2-1179-4030-9ab3-c5e476b0135b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:54.903651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:54.909811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374310b-11eb-4b6b-aca3-c413cbf8a665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:54.911046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374310b-11eb-4b6b-aca3-c413cbf8a665 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:54.911562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374310b-11eb-4b6b-aca3-c413cbf8a665 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:54.912106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374310b-11eb-4b6b-aca3-c413cbf8a665 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:54.913085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374310b-11eb-4b6b-aca3-c413cbf8a665 None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:54.913730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374310b-11eb-4b6b-aca3-c413cbf8a665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:54.915296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374310b-11eb-4b6b-aca3-c413cbf8a665 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:54.925249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 134/277] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:54.926178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374310b-11eb-4b6b-aca3-c413cbf8a665 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:55.023093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa5fc2-1179-4030-9ab3-c5e476b0135b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:55.037982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374310b-11eb-4b6b-aca3-c413cbf8a665 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:55.060145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edf887-3fa7-4d5b-8bfe-0822d74c01ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:55.060789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edf887-3fa7-4d5b-8bfe-0822d74c01ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.061068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edf887-3fa7-4d5b-8bfe-0822d74c01ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.061344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edf887-3fa7-4d5b-8bfe-0822d74c01ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.063056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edf887-3fa7-4d5b-8bfe-0822d74c01ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:55.064496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edf887-3fa7-4d5b-8bfe-0822d74c01ef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:55.066231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 144/278] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:55.066931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edf887-3fa7-4d5b-8bfe-0822d74c01ef None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:55.082137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39872e63-eee9-4b7b-ae12-bfbf09534553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:55.082813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39872e63-eee9-4b7b-ae12-bfbf09534553 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.083091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39872e63-eee9-4b7b-ae12-bfbf09534553 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.083360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39872e63-eee9-4b7b-ae12-bfbf09534553 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.095392 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39872e63-eee9-4b7b-ae12-bfbf09534553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-39872e63-eee9-4b7b-ae12-bfbf09534553 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:55.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:55.101229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:55.102033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 135/279] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:55.102738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39872e63-eee9-4b7b-ae12-bfbf09534553 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:55.118991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:55.160098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:55.180191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edf887-3fa7-4d5b-8bfe-0822d74c01ef None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:55.213040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39872e63-eee9-4b7b-ae12-bfbf09534553 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:55.220271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52102c97-7e34-4513-ae8b-cd160475f1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:55.221313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52102c97-7e34-4513-ae8b-cd160475f1bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.221598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52102c97-7e34-4513-ae8b-cd160475f1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.221871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52102c97-7e34-4513-ae8b-cd160475f1bc None admin] PATH_INFO: `/v3/users/acae83c400f34353bf787e6d9ddfbe7d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.238360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52102c97-7e34-4513-ae8b-cd160475f1bc None admin] RBAC: Authorizing `identity:get_user(user_id=acae83c400f34353bf787e6d9ddfbe7d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:55.240278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52102c97-7e34-4513-ae8b-cd160475f1bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:55.247152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 145/280] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:55 2022] GET /identity/v3/users/acae83c400f34353bf787e6d9ddfbe7d => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:55.247152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52102c97-7e34-4513-ae8b-cd160475f1bc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:55.251757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e30d6-2dcc-4762-a2f4-78fd449b5273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:55.252380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e30d6-2dcc-4762-a2f4-78fd449b5273 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.252660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e30d6-2dcc-4762-a2f4-78fd449b5273 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.252934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e30d6-2dcc-4762-a2f4-78fd449b5273 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.253625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e30d6-2dcc-4762-a2f4-78fd449b5273 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:55.255132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e30d6-2dcc-4762-a2f4-78fd449b5273 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:55.278041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 136/281] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:44:55 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:55.286735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e30d6-2dcc-4762-a2f4-78fd449b5273 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:55.365495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:55.369403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52102c97-7e34-4513-ae8b-cd160475f1bc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:55.413897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227f521-991f-4b5c-b186-d4b428b0042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:55.414683 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227f521-991f-4b5c-b186-d4b428b0042c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.415033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227f521-991f-4b5c-b186-d4b428b0042c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.415460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227f521-991f-4b5c-b186-d4b428b0042c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.416318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227f521-991f-4b5c-b186-d4b428b0042c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:55.416701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227f521-991f-4b5c-b186-d4b428b0042c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:55.418175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227f521-991f-4b5c-b186-d4b428b0042c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:55.418319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e30d6-2dcc-4762-a2f4-78fd449b5273 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:55.429238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:55.430486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 146/282] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:55.438478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227f521-991f-4b5c-b186-d4b428b0042c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:55.462416 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61113986-ee73-4dd1-8cd2-a2745f1488b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:55.463064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61113986-ee73-4dd1-8cd2-a2745f1488b4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.463339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61113986-ee73-4dd1-8cd2-a2745f1488b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.463608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61113986-ee73-4dd1-8cd2-a2745f1488b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.464396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61113986-ee73-4dd1-8cd2-a2745f1488b4 None admin] RBAC: Adding query filter params (role.id=3d81447d2d66443384ce9ae7dd5ea35d, user.id=acae83c400f34353bf787e6d9ddfbe7d, scope.system=all) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:55.464707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61113986-ee73-4dd1-8cd2-a2745f1488b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:55.466152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61113986-ee73-4dd1-8cd2-a2745f1488b4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:55.472543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 137/283] 10.222.0.20 () {56 vars in 1308 bytes} [Sat Jul 30 04:44:55 2022] GET /identity/v3/role_assignments?user.id=acae83c400f34353bf787e6d9ddfbe7d&scope.system=all&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:55.567357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227f521-991f-4b5c-b186-d4b428b0042c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:55.613649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c753977-2e64-4428-9c40-9f1cca39f8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:55.614402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c753977-2e64-4428-9c40-9f1cca39f8bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.614764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c753977-2e64-4428-9c40-9f1cca39f8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.615043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c753977-2e64-4428-9c40-9f1cca39f8bf None admin] PATH_INFO: `/v3/users/7b8f645f0e8449cf907423d22643a5de` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.616306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c753977-2e64-4428-9c40-9f1cca39f8bf None admin] RBAC: Authorizing `identity:get_user(user_id=7b8f645f0e8449cf907423d22643a5de)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:55.617860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c753977-2e64-4428-9c40-9f1cca39f8bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:55.619765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 147/284] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:55 2022] GET /identity/v3/users/7b8f645f0e8449cf907423d22643a5de => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:55.625932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61113986-ee73-4dd1-8cd2-a2745f1488b4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:55.747104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61113986-ee73-4dd1-8cd2-a2745f1488b4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:55.786864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f22e61-3334-4403-8072-78857eac9d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:55.787904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f22e61-3334-4403-8072-78857eac9d93 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.788405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f22e61-3334-4403-8072-78857eac9d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.788876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f22e61-3334-4403-8072-78857eac9d93 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.790271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f22e61-3334-4403-8072-78857eac9d93 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:55.791951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f22e61-3334-4403-8072-78857eac9d93 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:55.794128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 138/285] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:55 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:55.800675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c753977-2e64-4428-9c40-9f1cca39f8bf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:55.862029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:55.898084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f6685-9791-4662-820f-c15017ae47c7 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.898737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f6685-9791-4662-820f-c15017ae47c7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.899215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f6685-9791-4662-820f-c15017ae47c7 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.901033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 139/286] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:44:55.908037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6604-0452-4f28-b69c-83158c0163a5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:55.908581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6604-0452-4f28-b69c-83158c0163a5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:55.909071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6604-0452-4f28-b69c-83158c0163a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:55.934791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-624b6604-0452-4f28-b69c-83158c0163a5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:55.936109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-624b6604-0452-4f28-b69c-83158c0163a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:55.969281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c753977-2e64-4428-9c40-9f1cca39f8bf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:55.996652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:56.021979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3cb9b-7553-4c7b-bb07-df75aafd6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:56.026629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3cb9b-7553-4c7b-bb07-df75aafd6164 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:56.026629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3cb9b-7553-4c7b-bb07-df75aafd6164 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:56.026629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3cb9b-7553-4c7b-bb07-df75aafd6164 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/users/7b8f645f0e8449cf907423d22643a5de/roles/6cce74d600ba46a2b7786018d0836956` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:56.027026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3cb9b-7553-4c7b-bb07-df75aafd6164 None admin] RBAC: Authorizing `identity:create_grant(project_id=733b6ca2bdf14132a5de09c5371d80ae, user_id=7b8f645f0e8449cf907423d22643a5de, role_id=6cce74d600ba46a2b7786018d0836956)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:56.028479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3cb9b-7553-4c7b-bb07-df75aafd6164 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:56.054487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 140/287] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:56.063109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 148/288] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:44:55 2022] PUT /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/users/7b8f645f0e8449cf907423d22643a5de/roles/6cce74d600ba46a2b7786018d0836956 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:44:56.105461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b6604-0452-4f28-b69c-83158c0163a5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:56.215662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b6604-0452-4f28-b69c-83158c0163a5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:56.257151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91668639-6369-462d-9bbf-7dad547190ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:56.257821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668639-6369-462d-9bbf-7dad547190ab None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:56.258100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668639-6369-462d-9bbf-7dad547190ab None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:56.258368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91668639-6369-462d-9bbf-7dad547190ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:56.281321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91668639-6369-462d-9bbf-7dad547190ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-91668639-6369-462d-9bbf-7dad547190ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:56.286267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:56.287203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:56.287989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 141/289] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:56.294632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3cb9b-7553-4c7b-bb07-df75aafd6164 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:56.403847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:56.414807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3cb9b-7553-4c7b-bb07-df75aafd6164 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:56.451471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51b55a-70da-4a76-9b22-e4431928520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:56.452646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51b55a-70da-4a76-9b22-e4431928520e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:56.452920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51b55a-70da-4a76-9b22-e4431928520e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:56.453191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51b55a-70da-4a76-9b22-e4431928520e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:56.453919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51b55a-70da-4a76-9b22-e4431928520e None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:56.454223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51b55a-70da-4a76-9b22-e4431928520e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:56.455646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51b55a-70da-4a76-9b22-e4431928520e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:56.465026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 149/290] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:56.470604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91668639-6369-462d-9bbf-7dad547190ab None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:56.502931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:56.575666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91668639-6369-462d-9bbf-7dad547190ab None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:56.610960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4923955-6205-4575-86a9-8e4122cb5a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:56.611611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4923955-6205-4575-86a9-8e4122cb5a9e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:56.611890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4923955-6205-4575-86a9-8e4122cb5a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:56.612170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4923955-6205-4575-86a9-8e4122cb5a9e None admin] PATH_INFO: `/v3/users/46298a6053564039a703effda1c0020d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:56.613371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4923955-6205-4575-86a9-8e4122cb5a9e None admin] RBAC: Authorizing `identity:get_user(user_id=46298a6053564039a703effda1c0020d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:56.614854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4923955-6205-4575-86a9-8e4122cb5a9e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:56.616753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 142/291] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:56 2022] GET /identity/v3/users/46298a6053564039a703effda1c0020d => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:56.622208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51b55a-70da-4a76-9b22-e4431928520e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:56.732290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51b55a-70da-4a76-9b22-e4431928520e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:56.769261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b2d0-ed77-41fd-94cb-cc3428c64f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:56.769902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b2d0-ed77-41fd-94cb-cc3428c64f8c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:56.770186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b2d0-ed77-41fd-94cb-cc3428c64f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:56.770481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4b2d0-ed77-41fd-94cb-cc3428c64f8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:56.793665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:56.845382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b2d0-ed77-41fd-94cb-cc3428c64f8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:56.846826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4b2d0-ed77-41fd-94cb-cc3428c64f8c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:56.956332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 150/292] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:56.961842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4923955-6205-4575-86a9-8e4122cb5a9e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:57.064395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:57.068708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4923955-6205-4575-86a9-8e4122cb5a9e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:57.104635 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d739c0b-bff2-4c41-9fe5-bdbca6b063ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:57.105202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d739c0b-bff2-4c41-9fe5-bdbca6b063ff None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:57.105484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d739c0b-bff2-4c41-9fe5-bdbca6b063ff None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:57.105756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d739c0b-bff2-4c41-9fe5-bdbca6b063ff None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:57.106957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d739c0b-bff2-4c41-9fe5-bdbca6b063ff None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:57.109206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d739c0b-bff2-4c41-9fe5-bdbca6b063ff None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:57.111431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 143/293] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:56 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:57.117301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b2d0-ed77-41fd-94cb-cc3428c64f8c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:57.174349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:57.222004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4b2d0-ed77-41fd-94cb-cc3428c64f8c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:57.257674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccebc0a-7f30-4ea9-aab4-59d7b38c2faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:57.258302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccebc0a-7f30-4ea9-aab4-59d7b38c2faa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:57.258689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccebc0a-7f30-4ea9-aab4-59d7b38c2faa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:57.258971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccebc0a-7f30-4ea9-aab4-59d7b38c2faa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:57.259760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccebc0a-7f30-4ea9-aab4-59d7b38c2faa None admin] RBAC: Adding query filter params (role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae, user.id=46298a6053564039a703effda1c0020d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:57.260070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccebc0a-7f30-4ea9-aab4-59d7b38c2faa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:57.261517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccebc0a-7f30-4ea9-aab4-59d7b38c2faa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:57.268379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 151/294] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:44:57 2022] GET /identity/v3/role_assignments?user.id=46298a6053564039a703effda1c0020d&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:57.919730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b4f22-06d2-402b-8ec5-e112c6b0facc None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:57.920102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b4f22-06d2-402b-8ec5-e112c6b0facc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:57.920289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b4f22-06d2-402b-8ec5-e112c6b0facc None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:57.922485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 144/295] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:57.927477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd454d4-c495-41a8-9547-c5053a6bc39d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:57.927925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd454d4-c495-41a8-9547-c5053a6bc39d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:57.928094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd454d4-c495-41a8-9547-c5053a6bc39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:57.947863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4dd454d4-c495-41a8-9547-c5053a6bc39d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:57.949066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4dd454d4-c495-41a8-9547-c5053a6bc39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:58.005477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593e87-9f67-4c3f-9057-e08a5b05d64f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.005477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593e87-9f67-4c3f-9057-e08a5b05d64f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.005477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593e87-9f67-4c3f-9057-e08a5b05d64f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.007404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 145/296] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:58.011822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d3c25-d9b0-480b-852f-e3a60291df24 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.012098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d3c25-d9b0-480b-852f-e3a60291df24 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.012368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d3c25-d9b0-480b-852f-e3a60291df24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.020168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 152/297] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:58.025306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dbf3b-afbd-43b8-8080-74863217158c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.025833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dbf3b-afbd-43b8-8080-74863217158c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.027000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dbf3b-afbd-43b8-8080-74863217158c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.029081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 153/298] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:44:58.035814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-930d3c25-d9b0-480b-852f-e3a60291df24 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:58.035814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-930d3c25-d9b0-480b-852f-e3a60291df24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:58.036009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518e8b8-8e1a-460a-9ba4-102de36e721e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.036009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518e8b8-8e1a-460a-9ba4-102de36e721e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.036009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518e8b8-8e1a-460a-9ba4-102de36e721e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.055314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e518e8b8-8e1a-460a-9ba4-102de36e721e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:58.056498 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e518e8b8-8e1a-460a-9ba4-102de36e721e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:58.101865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:58.108717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 146/299] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:58.109818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d3c25-d9b0-480b-852f-e3a60291df24 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.138479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 154/300] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:44:58.140591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518e8b8-8e1a-460a-9ba4-102de36e721e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.217461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:58.226134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d3c25-d9b0-480b-852f-e3a60291df24 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:58.249419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518e8b8-8e1a-460a-9ba4-102de36e721e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:58.267617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96adb4-bb9b-47cb-9d2a-c25a0acee527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:58.268509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96adb4-bb9b-47cb-9d2a-c25a0acee527 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.268963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96adb4-bb9b-47cb-9d2a-c25a0acee527 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.269431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96adb4-bb9b-47cb-9d2a-c25a0acee527 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.283261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96adb4-bb9b-47cb-9d2a-c25a0acee527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:58.283714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17298234-f20c-4a5a-b278-c0a79510b6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:58.284350 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17298234-f20c-4a5a-b278-c0a79510b6ae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.284627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17298234-f20c-4a5a-b278-c0a79510b6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.284896 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17298234-f20c-4a5a-b278-c0a79510b6ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3a96adb4-bb9b-47cb-9d2a-c25a0acee527 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:58.290017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:58.290941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:58.291750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:44:58.297723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 147/301] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:58.297723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96adb4-bb9b-47cb-9d2a-c25a0acee527 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.301377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17298234-f20c-4a5a-b278-c0a79510b6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-17298234-f20c-4a5a-b278-c0a79510b6ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:58.306397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:58.307285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:58.308077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 155/302] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:58.308718 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17298234-f20c-4a5a-b278-c0a79510b6ae None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.348773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:58.411286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96adb4-bb9b-47cb-9d2a-c25a0acee527 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:58.420897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17298234-f20c-4a5a-b278-c0a79510b6ae None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:58.449664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fad7b-95f0-4382-950f-dd7fdcd93dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:58.450309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fad7b-95f0-4382-950f-dd7fdcd93dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.450643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fad7b-95f0-4382-950f-dd7fdcd93dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.450895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fad7b-95f0-4382-950f-dd7fdcd93dd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.452711 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fad7b-95f0-4382-950f-dd7fdcd93dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:58.454297 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fad7b-95f0-4382-950f-dd7fdcd93dd1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:58.456128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 148/303] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:44:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:58.456958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bae457-27d8-49c2-adac-6626c4cc2a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:58.457581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bae457-27d8-49c2-adac-6626c4cc2a28 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.457861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bae457-27d8-49c2-adac-6626c4cc2a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.458134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bae457-27d8-49c2-adac-6626c4cc2a28 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.458890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bae457-27d8-49c2-adac-6626c4cc2a28 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:58.459193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bae457-27d8-49c2-adac-6626c4cc2a28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:58.460817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bae457-27d8-49c2-adac-6626c4cc2a28 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:58.461234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fad7b-95f0-4382-950f-dd7fdcd93dd1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.469816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 156/304] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:58.471591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bae457-27d8-49c2-adac-6626c4cc2a28 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.488078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:58.569072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fad7b-95f0-4382-950f-dd7fdcd93dd1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:58.575918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:58.581935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bae457-27d8-49c2-adac-6626c4cc2a28 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:58.606378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dde732-2c07-4c5e-9a05-146847bd8758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:58.607021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dde732-2c07-4c5e-9a05-146847bd8758 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.607300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dde732-2c07-4c5e-9a05-146847bd8758 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.607568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dde732-2c07-4c5e-9a05-146847bd8758 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.608558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dde732-2c07-4c5e-9a05-146847bd8758 None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:58.608975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dde732-2c07-4c5e-9a05-146847bd8758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:58.610364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dde732-2c07-4c5e-9a05-146847bd8758 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:58.616921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87313f-f75c-49f0-ad97-fafcb5122174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:58.617545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87313f-f75c-49f0-ad97-fafcb5122174 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.617821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87313f-f75c-49f0-ad97-fafcb5122174 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.618095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87313f-f75c-49f0-ad97-fafcb5122174 None admin] PATH_INFO: `/v3/groups` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.618795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87313f-f75c-49f0-ad97-fafcb5122174 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:58.619903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 149/305] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:58.620208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87313f-f75c-49f0-ad97-fafcb5122174 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:58.620993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dde732-2c07-4c5e-9a05-146847bd8758 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.641081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 157/306] 10.222.0.20 () {60 vars in 1134 bytes} [Sat Jul 30 04:44:58 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:44:58.641904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87313f-f75c-49f0-ad97-fafcb5122174 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.750902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:58.777047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dde732-2c07-4c5e-9a05-146847bd8758 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:58.790326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87313f-f75c-49f0-ad97-fafcb5122174 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:58.827894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59766ff7-29c6-449d-8be6-f33b8173c654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:58.828535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59766ff7-29c6-449d-8be6-f33b8173c654 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.828816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59766ff7-29c6-449d-8be6-f33b8173c654 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.829088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59766ff7-29c6-449d-8be6-f33b8173c654 None admin] PATH_INFO: `/v3/users/7176bb3b038e41198f73931b761a98b0` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.832594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3004f0-0ff6-4faf-8f15-b98bc067aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:58.833242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3004f0-0ff6-4faf-8f15-b98bc067aeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.833518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3004f0-0ff6-4faf-8f15-b98bc067aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.833785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3004f0-0ff6-4faf-8f15-b98bc067aeb0 None admin] PATH_INFO: `/v3/users/acae83c400f34353bf787e6d9ddfbe7d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.834995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3004f0-0ff6-4faf-8f15-b98bc067aeb0 None admin] RBAC: Authorizing `identity:get_user(user_id=acae83c400f34353bf787e6d9ddfbe7d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:58.836436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3004f0-0ff6-4faf-8f15-b98bc067aeb0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:58.838342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 158/307] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:58 2022] GET /identity/v3/users/acae83c400f34353bf787e6d9ddfbe7d => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:58.839515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658efabf-66d7-4850-9168-ad47150e6618 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.839798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658efabf-66d7-4850-9168-ad47150e6618 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.840070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658efabf-66d7-4850-9168-ad47150e6618 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.843426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 159/308] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:44:58.844407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658efabf-66d7-4850-9168-ad47150e6618 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.849653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59766ff7-29c6-449d-8be6-f33b8173c654 None admin] RBAC: Authorizing `identity:get_user(user_id=7176bb3b038e41198f73931b761a98b0)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:58.851969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59766ff7-29c6-449d-8be6-f33b8173c654 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:58.858050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 150/309] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:58 2022] GET /identity/v3/users/7176bb3b038e41198f73931b761a98b0 => generated 293 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:58.858050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefab8f5-ff09-47d8-b800-95f329f85900 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:58.858050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefab8f5-ff09-47d8-b800-95f329f85900 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:58.858050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefab8f5-ff09-47d8-b800-95f329f85900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:58.875854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:58.877457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-eefab8f5-ff09-47d8-b800-95f329f85900 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:44:58.878744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eefab8f5-ff09-47d8-b800-95f329f85900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:44:58.953312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 151/310] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:44:58.954560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefab8f5-ff09-47d8-b800-95f329f85900 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:58.971881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658efabf-66d7-4850-9168-ad47150e6618 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:59.011384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1985b4-ac43-41fc-b31c-7cb9d34ed4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:59.011984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1985b4-ac43-41fc-b31c-7cb9d34ed4e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:59.012275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1985b4-ac43-41fc-b31c-7cb9d34ed4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:59.012557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1985b4-ac43-41fc-b31c-7cb9d34ed4e9 None admin] PATH_INFO: `/v3/system/users/acae83c400f34353bf787e6d9ddfbe7d/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:59.014036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1985b4-ac43-41fc-b31c-7cb9d34ed4e9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=acae83c400f34353bf787e6d9ddfbe7d, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:59.015489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1985b4-ac43-41fc-b31c-7cb9d34ed4e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:59.028037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 160/311] 10.222.0.20 () {58 vars in 1334 bytes} [Sat Jul 30 04:44:58 2022] PUT /identity/v3/system/users/acae83c400f34353bf787e6d9ddfbe7d/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:44:59.028709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1985b4-ac43-41fc-b31c-7cb9d34ed4e9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:59.059405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:59.100983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefab8f5-ff09-47d8-b800-95f329f85900 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:59.150263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:59.151804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4cd06-ab48-4b14-ab82-a014e70e5315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:59.152955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4cd06-ab48-4b14-ab82-a014e70e5315 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:59.153303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4cd06-ab48-4b14-ab82-a014e70e5315 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:59.153578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4cd06-ab48-4b14-ab82-a014e70e5315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:59.154752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4cd06-ab48-4b14-ab82-a014e70e5315 None admin] RBAC: Adding query filter params (role.id=6cce74d600ba46a2b7786018d0836956, user.id=7176bb3b038e41198f73931b761a98b0, scope.system=all) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:59.155599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4cd06-ab48-4b14-ab82-a014e70e5315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:59.158804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4cd06-ab48-4b14-ab82-a014e70e5315 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:59.167237 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1985b4-ac43-41fc-b31c-7cb9d34ed4e9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:59.167546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 152/312] 10.222.0.20 () {56 vars in 1308 bytes} [Sat Jul 30 04:44:58 2022] GET /identity/v3/role_assignments?user.id=7176bb3b038e41198f73931b761a98b0&scope.system=all&role.id=6cce74d600ba46a2b7786018d0836956 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:59.217286 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33cca9-2c90-4166-be0f-6e7b2b4daae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:59.218519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33cca9-2c90-4166-be0f-6e7b2b4daae8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:59.218829 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33cca9-2c90-4166-be0f-6e7b2b4daae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:59.219092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33cca9-2c90-4166-be0f-6e7b2b4daae8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:59.231070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33cca9-2c90-4166-be0f-6e7b2b4daae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0d33cca9-2c90-4166-be0f-6e7b2b4daae8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:44:59.237020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:44:59.237989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:44:59.238844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 161/313] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:44:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:44:59.244273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4cd06-ab48-4b14-ab82-a014e70e5315 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:59.355506 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4cd06-ab48-4b14-ab82-a014e70e5315 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:59.391679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3876c3-cd8e-454a-b127-965d470f0f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:59.392295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3876c3-cd8e-454a-b127-965d470f0f63 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:59.392570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3876c3-cd8e-454a-b127-965d470f0f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:59.392834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3876c3-cd8e-454a-b127-965d470f0f63 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:59.393552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3876c3-cd8e-454a-b127-965d470f0f63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:44:59.393846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3876c3-cd8e-454a-b127-965d470f0f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:59.395244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3876c3-cd8e-454a-b127-965d470f0f63 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:59.404301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 153/314] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:44:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:44:59.406799 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:59.410167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33cca9-2c90-4166-be0f-6e7b2b4daae8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:59.518631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33cca9-2c90-4166-be0f-6e7b2b4daae8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:59.554759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec296a1d-a7a4-4725-b086-30f16d02185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:59.555405 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec296a1d-a7a4-4725-b086-30f16d02185d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:59.555682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec296a1d-a7a4-4725-b086-30f16d02185d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:59.555958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec296a1d-a7a4-4725-b086-30f16d02185d None admin] PATH_INFO: `/v3/users/7b8f645f0e8449cf907423d22643a5de` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:59.557129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec296a1d-a7a4-4725-b086-30f16d02185d None admin] RBAC: Authorizing `identity:get_user(user_id=7b8f645f0e8449cf907423d22643a5de)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:59.558596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec296a1d-a7a4-4725-b086-30f16d02185d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:59.560365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 162/315] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:44:59 2022] GET /identity/v3/users/7b8f645f0e8449cf907423d22643a5de => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:44:59.565983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3876c3-cd8e-454a-b127-965d470f0f63 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:59.671287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3876c3-cd8e-454a-b127-965d470f0f63 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:44:59.707896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02b18a-8f16-4143-af89-b3b5caa2989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:44:59.708566 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02b18a-8f16-4143-af89-b3b5caa2989f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:44:59.708845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02b18a-8f16-4143-af89-b3b5caa2989f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:44:59.709116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02b18a-8f16-4143-af89-b3b5caa2989f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:44:59.778103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:44:59.780406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02b18a-8f16-4143-af89-b3b5caa2989f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:44:59.782115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02b18a-8f16-4143-af89-b3b5caa2989f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:44:59.887109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 154/316] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:44:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:44:59.893410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec296a1d-a7a4-4725-b086-30f16d02185d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:44:59.914476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:00.006470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec296a1d-a7a4-4725-b086-30f16d02185d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:00.042808 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f42cec-4d41-4db3-8f38-addae988c7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:00.043510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f42cec-4d41-4db3-8f38-addae988c7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:00.043797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f42cec-4d41-4db3-8f38-addae988c7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:00.044074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f42cec-4d41-4db3-8f38-addae988c7f6 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:00.045245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f42cec-4d41-4db3-8f38-addae988c7f6 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:00.047479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f42cec-4d41-4db3-8f38-addae988c7f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:00.049461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 163/317] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:44:59 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:00.055068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02b18a-8f16-4143-af89-b3b5caa2989f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:00.152757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:00.156635 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02b18a-8f16-4143-af89-b3b5caa2989f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:00.193130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354dd3a-73c2-41d2-9e0c-6237660a0fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:00.193760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354dd3a-73c2-41d2-9e0c-6237660a0fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:00.194033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354dd3a-73c2-41d2-9e0c-6237660a0fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:00.194344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354dd3a-73c2-41d2-9e0c-6237660a0fe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:00.195177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354dd3a-73c2-41d2-9e0c-6237660a0fe3 None admin] RBAC: Adding query filter params (role.id=6cce74d600ba46a2b7786018d0836956, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae, user.id=7b8f645f0e8449cf907423d22643a5de) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:00.195487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354dd3a-73c2-41d2-9e0c-6237660a0fe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:00.196941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354dd3a-73c2-41d2-9e0c-6237660a0fe3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:00.210586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 155/318] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:00 2022] GET /identity/v3/role_assignments?user.id=7b8f645f0e8449cf907423d22643a5de&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=6cce74d600ba46a2b7786018d0836956 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:00.783518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f418e-fd62-4675-a3db-d23ee9de9ce0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:00.783901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f418e-fd62-4675-a3db-d23ee9de9ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:00.784105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f418e-fd62-4675-a3db-d23ee9de9ce0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:00.785581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 164/319] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:00.790606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a434f0c-0b2f-4333-9833-88d6a7a6a635 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:00.790898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a434f0c-0b2f-4333-9833-88d6a7a6a635 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:00.791173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a434f0c-0b2f-4333-9833-88d6a7a6a635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:00.810179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6a434f0c-0b2f-4333-9833-88d6a7a6a635 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:00.811481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6a434f0c-0b2f-4333-9833-88d6a7a6a635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:00.881055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 156/320] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:00.903746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f418e-fd62-4675-a3db-d23ee9de9ce0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:00.993620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:01.015717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f418e-fd62-4675-a3db-d23ee9de9ce0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:01.054931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d39e0-3430-47ec-bbed-9678bedceef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:01.055501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d39e0-3430-47ec-bbed-9678bedceef4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.055783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d39e0-3430-47ec-bbed-9678bedceef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.056059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d39e0-3430-47ec-bbed-9678bedceef4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.057203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d39e0-3430-47ec-bbed-9678bedceef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:01.058695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d39e0-3430-47ec-bbed-9678bedceef4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:01.060494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 165/321] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:01.066687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a434f0c-0b2f-4333-9833-88d6a7a6a635 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:01.109941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c70f10-9eaa-4218-92ab-4b368801898e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.110280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c70f10-9eaa-4218-92ab-4b368801898e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.110527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c70f10-9eaa-4218-92ab-4b368801898e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.111891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 166/322] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:45:01.116709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c69cf-c4a4-42b5-aff4-5a9beb021540 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.116998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c69cf-c4a4-42b5-aff4-5a9beb021540 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.117287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c69cf-c4a4-42b5-aff4-5a9beb021540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.137109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-567c69cf-c4a4-42b5-aff4-5a9beb021540 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:01.138309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-567c69cf-c4a4-42b5-aff4-5a9beb021540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:01.164607 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:01.174189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a434f0c-0b2f-4333-9833-88d6a7a6a635 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:01.211055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09597ba-5657-465a-86a5-2bcbd603b19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:01.211683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09597ba-5657-465a-86a5-2bcbd603b19f None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.211962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09597ba-5657-465a-86a5-2bcbd603b19f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.214471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09597ba-5657-465a-86a5-2bcbd603b19f None admin] PATH_INFO: `/v3/groups` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.214613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 167/323] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:01.214716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09597ba-5657-465a-86a5-2bcbd603b19f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:01.214716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09597ba-5657-465a-86a5-2bcbd603b19f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:01.228976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddb7fb-c41e-42a2-9c54-112433c1634f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.229484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddb7fb-c41e-42a2-9c54-112433c1634f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.229588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddb7fb-c41e-42a2-9c54-112433c1634f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.231058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 168/324] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:01.235979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13605db-2154-4135-b119-303ae29a776d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.236265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13605db-2154-4135-b119-303ae29a776d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.236545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13605db-2154-4135-b119-303ae29a776d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.242633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 157/325] 10.222.0.20 () {60 vars in 1134 bytes} [Sat Jul 30 04:45:01 2022] POST /identity/v3/groups => generated 226 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:45:01.248681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09597ba-5657-465a-86a5-2bcbd603b19f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:01.256667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d13605db-2154-4135-b119-303ae29a776d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:01.257901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d13605db-2154-4135-b119-303ae29a776d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:01.334645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 169/326] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:01.357341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13605db-2154-4135-b119-303ae29a776d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:01.361425 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09597ba-5657-465a-86a5-2bcbd603b19f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:01.375424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:01.409852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c43052-f543-4258-b8a6-7277e35ee54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:01.410542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c43052-f543-4258-b8a6-7277e35ee54b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.410809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c43052-f543-4258-b8a6-7277e35ee54b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.411087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c43052-f543-4258-b8a6-7277e35ee54b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.423294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c43052-f543-4258-b8a6-7277e35ee54b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-15c43052-f543-4258-b8a6-7277e35ee54b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:01.429292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:01.430360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:01.431422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:01.436722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 158/327] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:01.437605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c43052-f543-4258-b8a6-7277e35ee54b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:01.453903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:01.488709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13605db-2154-4135-b119-303ae29a776d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:01.534743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f89f92-ae9a-4e8c-ad05-fb87ca280b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:01.534743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f89f92-ae9a-4e8c-ad05-fb87ca280b20 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.535002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f89f92-ae9a-4e8c-ad05-fb87ca280b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.535097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f89f92-ae9a-4e8c-ad05-fb87ca280b20 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.548577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f89f92-ae9a-4e8c-ad05-fb87ca280b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d3f89f92-ae9a-4e8c-ad05-fb87ca280b20 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:01.554411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:01.555324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:01.556280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 170/328] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:01.562464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f89f92-ae9a-4e8c-ad05-fb87ca280b20 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:01.576516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c43052-f543-4258-b8a6-7277e35ee54b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:01.620519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c2b0-a794-4e7c-9fdb-1189179c3c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:01.621327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c2b0-a794-4e7c-9fdb-1189179c3c21 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.622166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c2b0-a794-4e7c-9fdb-1189179c3c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.622166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c2b0-a794-4e7c-9fdb-1189179c3c21 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.622890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c2b0-a794-4e7c-9fdb-1189179c3c21 None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:01.623298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c2b0-a794-4e7c-9fdb-1189179c3c21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:01.625356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c2b0-a794-4e7c-9fdb-1189179c3c21 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:01.635857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 159/329] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:45:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:01.642151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c2b0-a794-4e7c-9fdb-1189179c3c21 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:01.665887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:01.698993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f89f92-ae9a-4e8c-ad05-fb87ca280b20 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:01.742321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:01.751471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d75e42-7c98-4eab-af5a-99fa17916749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:01.752798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d75e42-7c98-4eab-af5a-99fa17916749 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.753188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d75e42-7c98-4eab-af5a-99fa17916749 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.753713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d75e42-7c98-4eab-af5a-99fa17916749 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.754672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d75e42-7c98-4eab-af5a-99fa17916749 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:01.755324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d75e42-7c98-4eab-af5a-99fa17916749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:01.757243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d75e42-7c98-4eab-af5a-99fa17916749 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:01.770828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 171/330] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:45:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:01.776412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d75e42-7c98-4eab-af5a-99fa17916749 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:01.777048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c2b0-a794-4e7c-9fdb-1189179c3c21 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:01.818018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c0ce8-d4bb-4bc6-8d7e-8b166abb3f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:01.818702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c0ce8-d4bb-4bc6-8d7e-8b166abb3f8f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.818976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c0ce8-d4bb-4bc6-8d7e-8b166abb3f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.819246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c0ce8-d4bb-4bc6-8d7e-8b166abb3f8f None admin] PATH_INFO: `/v3/users/acae83c400f34353bf787e6d9ddfbe7d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.820427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c0ce8-d4bb-4bc6-8d7e-8b166abb3f8f None admin] RBAC: Authorizing `identity:get_user(user_id=acae83c400f34353bf787e6d9ddfbe7d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:01.821900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c0ce8-d4bb-4bc6-8d7e-8b166abb3f8f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:01.823987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 160/331] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:45:01 2022] GET /identity/v3/users/acae83c400f34353bf787e6d9ddfbe7d => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:01.829987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c0ce8-d4bb-4bc6-8d7e-8b166abb3f8f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:01.894930 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d75e42-7c98-4eab-af5a-99fa17916749 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:01.935150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0411c-1676-4f19-956d-63d98707b0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:01.935901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0411c-1676-4f19-956d-63d98707b0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.936315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0411c-1676-4f19-956d-63d98707b0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.936745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0411c-1676-4f19-956d-63d98707b0e2 None admin] PATH_INFO: `/v3/users/7176bb3b038e41198f73931b761a98b0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.938120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0411c-1676-4f19-956d-63d98707b0e2 None admin] RBAC: Authorizing `identity:get_user(user_id=7176bb3b038e41198f73931b761a98b0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:01.939693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0411c-1676-4f19-956d-63d98707b0e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:01.942825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 172/332] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:45:01 2022] GET /identity/v3/users/7176bb3b038e41198f73931b761a98b0 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:01.948547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0411c-1676-4f19-956d-63d98707b0e2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:01.953210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:01.958192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c0ce8-d4bb-4bc6-8d7e-8b166abb3f8f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:01.995379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962126d-bf48-477a-8b7a-4bebaf0fb6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:01.995379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962126d-bf48-477a-8b7a-4bebaf0fb6db None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:01.996075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962126d-bf48-477a-8b7a-4bebaf0fb6db None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:01.996075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962126d-bf48-477a-8b7a-4bebaf0fb6db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:01.997480 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962126d-bf48-477a-8b7a-4bebaf0fb6db None admin] RBAC: Adding query filter params (role.id=3d81447d2d66443384ce9ae7dd5ea35d, user.id=acae83c400f34353bf787e6d9ddfbe7d, scope.system=all) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:01.997788 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962126d-bf48-477a-8b7a-4bebaf0fb6db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:02.000448 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962126d-bf48-477a-8b7a-4bebaf0fb6db None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:02.020255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 161/333] 10.222.0.20 () {56 vars in 1308 bytes} [Sat Jul 30 04:45:01 2022] GET /identity/v3/role_assignments?user.id=acae83c400f34353bf787e6d9ddfbe7d&scope.system=all&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:02.068800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0411c-1676-4f19-956d-63d98707b0e2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:02.107005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024405a2-c3ad-4e86-ab5d-bf170983ce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:02.107703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024405a2-c3ad-4e86-ab5d-bf170983ce98 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:02.107977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024405a2-c3ad-4e86-ab5d-bf170983ce98 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:02.108339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024405a2-c3ad-4e86-ab5d-bf170983ce98 None admin] PATH_INFO: `/v3/system/users/7176bb3b038e41198f73931b761a98b0/roles/6cce74d600ba46a2b7786018d0836956` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:02.110147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024405a2-c3ad-4e86-ab5d-bf170983ce98 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7176bb3b038e41198f73931b761a98b0, role_id=6cce74d600ba46a2b7786018d0836956)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:02.111611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024405a2-c3ad-4e86-ab5d-bf170983ce98 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:02.125526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 173/334] 10.222.0.20 () {58 vars in 1334 bytes} [Sat Jul 30 04:45:01 2022] PUT /identity/v3/system/users/7176bb3b038e41198f73931b761a98b0/roles/6cce74d600ba46a2b7786018d0836956 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:03.663277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f37118-70e0-4dc8-957c-957981c2a5e8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:03.664288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f37118-70e0-4dc8-957c-957981c2a5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:03.664790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f37118-70e0-4dc8-957c-957981c2a5e8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:03.666663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 162/335] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:03.672686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d2d54-342d-4dbd-a1ba-525682dbb971 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:03.673544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d2d54-342d-4dbd-a1ba-525682dbb971 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:03.674045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d2d54-342d-4dbd-a1ba-525682dbb971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:03.698982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-904d2d54-342d-4dbd-a1ba-525682dbb971 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:03.700502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-904d2d54-342d-4dbd-a1ba-525682dbb971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:03.780449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:03.792560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 174/336] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:03.830313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f37118-70e0-4dc8-957c-957981c2a5e8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:03.833699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:03.891468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116f014-b840-400b-b8fc-8a8bcc7f6d02 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:03.891793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116f014-b840-400b-b8fc-8a8bcc7f6d02 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:03.892021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116f014-b840-400b-b8fc-8a8bcc7f6d02 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:03.896975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 175/337] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:45:03.898429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98c845-a57e-4c5a-9c38-4106ad0311e1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:03.898743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98c845-a57e-4c5a-9c38-4106ad0311e1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:03.899021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98c845-a57e-4c5a-9c38-4106ad0311e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:03.938565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9f98c845-a57e-4c5a-9c38-4106ad0311e1 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:03.938565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9f98c845-a57e-4c5a-9c38-4106ad0311e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:03.976099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f37118-70e0-4dc8-957c-957981c2a5e8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:04.040130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691e09f-38ab-411f-b46a-227e38284c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:04.041117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691e09f-38ab-411f-b46a-227e38284c89 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:04.041604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691e09f-38ab-411f-b46a-227e38284c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:04.042097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691e09f-38ab-411f-b46a-227e38284c89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:04.053863 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691e09f-38ab-411f-b46a-227e38284c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d691e09f-38ab-411f-b46a-227e38284c89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:04.064559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:04.065492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:04.066306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 163/338] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:04.078649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691e09f-38ab-411f-b46a-227e38284c89 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:04.089770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 176/339] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:04.132100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98c845-a57e-4c5a-9c38-4106ad0311e1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:04.179025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:04.256879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691e09f-38ab-411f-b46a-227e38284c89 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:04.327382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a785d-f369-4696-9c14-5718fe277c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:04.328471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a785d-f369-4696-9c14-5718fe277c1e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:04.328763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a785d-f369-4696-9c14-5718fe277c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:04.329043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a785d-f369-4696-9c14-5718fe277c1e None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:04.329786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a785d-f369-4696-9c14-5718fe277c1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:04.330095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a785d-f369-4696-9c14-5718fe277c1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:04.336264 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a785d-f369-4696-9c14-5718fe277c1e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:04.347748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 164/340] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:45:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:04.353813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a785d-f369-4696-9c14-5718fe277c1e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:04.392722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98c845-a57e-4c5a-9c38-4106ad0311e1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:04.446824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:04.463361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085548dc-44b1-43f3-b3c3-b61d7c245de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:04.464115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085548dc-44b1-43f3-b3c3-b61d7c245de1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:04.464467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085548dc-44b1-43f3-b3c3-b61d7c245de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:04.464814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085548dc-44b1-43f3-b3c3-b61d7c245de1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:04.495147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085548dc-44b1-43f3-b3c3-b61d7c245de1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:04.499219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a785d-f369-4696-9c14-5718fe277c1e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-085548dc-44b1-43f3-b3c3-b61d7c245de1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:04.500827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:04.501715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:04.502556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 177/341] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:04.507994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47930-2ee4-4c1f-bd8f-0fcb2fb24765 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:04.508206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47930-2ee4-4c1f-bd8f-0fcb2fb24765 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:04.508679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a47930-2ee4-4c1f-bd8f-0fcb2fb24765 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:04.510081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 178/342] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:04.510878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47930-2ee4-4c1f-bd8f-0fcb2fb24765 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:04.528723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:04.542832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40150a3-2c97-4db9-85e7-4575c2a96452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:04.543466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40150a3-2c97-4db9-85e7-4575c2a96452 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:04.543751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40150a3-2c97-4db9-85e7-4575c2a96452 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:04.544030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40150a3-2c97-4db9-85e7-4575c2a96452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:04.682760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40150a3-2c97-4db9-85e7-4575c2a96452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:04.684234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40150a3-2c97-4db9-85e7-4575c2a96452 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:04.725852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a47930-2ee4-4c1f-bd8f-0fcb2fb24765 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:04.771264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb60998-541b-4642-8535-365511f379bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:04.771870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb60998-541b-4642-8535-365511f379bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:04.772151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb60998-541b-4642-8535-365511f379bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:04.772424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb60998-541b-4642-8535-365511f379bb None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:04.773144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb60998-541b-4642-8535-365511f379bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:04.773446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb60998-541b-4642-8535-365511f379bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:04.774845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb60998-541b-4642-8535-365511f379bb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:04.787845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 179/343] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:04.789116 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a991ca1-4742-4617-a20f-1eeb5190ab83 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:04.789404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a991ca1-4742-4617-a20f-1eeb5190ab83 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:04.789680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a991ca1-4742-4617-a20f-1eeb5190ab83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:04.810637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7a991ca1-4742-4617-a20f-1eeb5190ab83 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:04.812560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7a991ca1-4742-4617-a20f-1eeb5190ab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:04.861056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 165/344] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:04.862258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40150a3-2c97-4db9-85e7-4575c2a96452 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:04.876900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:04.904585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:04.927508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 180/345] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:04.928210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a991ca1-4742-4617-a20f-1eeb5190ab83 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:05.056637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40150a3-2c97-4db9-85e7-4575c2a96452 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:05.087929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a991ca1-4742-4617-a20f-1eeb5190ab83 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:05.137480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1cb0-e4f0-4ac4-9780-c0d509cdd76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:05.138514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1cb0-e4f0-4ac4-9780-c0d509cdd76d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.139004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1cb0-e4f0-4ac4-9780-c0d509cdd76d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.140020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b1cb0-e4f0-4ac4-9780-c0d509cdd76d None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.142358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b1cb0-e4f0-4ac4-9780-c0d509cdd76d None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:05.144184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b1cb0-e4f0-4ac4-9780-c0d509cdd76d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:05.146104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 181/346] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:04 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:05.147264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1cb0-e4f0-4ac4-9780-c0d509cdd76d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:05.168150 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5326031-d213-4fa3-a1ce-bdabb55f384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:05.168787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5326031-d213-4fa3-a1ce-bdabb55f384f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.169073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5326031-d213-4fa3-a1ce-bdabb55f384f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.169349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5326031-d213-4fa3-a1ce-bdabb55f384f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.212313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:05.300830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b1cb0-e4f0-4ac4-9780-c0d509cdd76d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:05.306034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5326031-d213-4fa3-a1ce-bdabb55f384f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:05.307568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5326031-d213-4fa3-a1ce-bdabb55f384f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:05.339127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:05.353523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f44fe-f8fd-4532-a157-2092e81c8bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:05.354428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f44fe-f8fd-4532-a157-2092e81c8bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.354906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f44fe-f8fd-4532-a157-2092e81c8bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.355365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f44fe-f8fd-4532-a157-2092e81c8bc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.370765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f44fe-f8fd-4532-a157-2092e81c8bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-710f44fe-f8fd-4532-a157-2092e81c8bc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:05.378340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:05.379591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:05.380549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:05.386802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 182/347] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:05.387886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f44fe-f8fd-4532-a157-2092e81c8bc3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:05.424568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 166/348] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:05.425854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973cb24-9476-4099-9093-51783a621488 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.426140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973cb24-9476-4099-9093-51783a621488 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.426483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973cb24-9476-4099-9093-51783a621488 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.431800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 167/349] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:05.431800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973cb24-9476-4099-9093-51783a621488 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:05.508387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f44fe-f8fd-4532-a157-2092e81c8bc3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:05.520474 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:05.544811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973cb24-9476-4099-9093-51783a621488 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:05.557604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4c9d7-9347-4836-bc0a-c7629d6ee841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:05.558241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4c9d7-9347-4836-bc0a-c7629d6ee841 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.558549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4c9d7-9347-4836-bc0a-c7629d6ee841 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.558873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4c9d7-9347-4836-bc0a-c7629d6ee841 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.567352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4c9d7-9347-4836-bc0a-c7629d6ee841 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:05.569403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4c9d7-9347-4836-bc0a-c7629d6ee841 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:05.571308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 183/350] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:05 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:05.571898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4c9d7-9347-4836-bc0a-c7629d6ee841 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:05.577297 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:05.589582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea43aab-12e6-492c-b918-61fcb2293c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:05.590257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea43aab-12e6-492c-b918-61fcb2293c9b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.590595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea43aab-12e6-492c-b918-61fcb2293c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.590868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea43aab-12e6-492c-b918-61fcb2293c9b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.591594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea43aab-12e6-492c-b918-61fcb2293c9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:05.592524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea43aab-12e6-492c-b918-61fcb2293c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:05.594555 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea43aab-12e6-492c-b918-61fcb2293c9b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:05.603225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 168/351] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:45:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:05.605587 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767af56-1b6b-4478-8daa-227941f4608f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.605874 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767af56-1b6b-4478-8daa-227941f4608f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.606146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767af56-1b6b-4478-8daa-227941f4608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.625484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1767af56-1b6b-4478-8daa-227941f4608f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:05.626758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1767af56-1b6b-4478-8daa-227941f4608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:05.718472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4c9d7-9347-4836-bc0a-c7629d6ee841 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:05.729543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 169/352] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:05.732363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cc455-fdfd-4796-a3d2-80b2ae6ce2f2 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.732860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cc455-fdfd-4796-a3d2-80b2ae6ce2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.734400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cc455-fdfd-4796-a3d2-80b2ae6ce2f2 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.736306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 170/353] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:05.737309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cc455-fdfd-4796-a3d2-80b2ae6ce2f2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:05.760986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5482aab-ac67-4b72-aa0d-b23ddb084cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:05.761724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5482aab-ac67-4b72-aa0d-b23ddb084cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.762258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5482aab-ac67-4b72-aa0d-b23ddb084cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.762606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5482aab-ac67-4b72-aa0d-b23ddb084cb9 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.763796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5482aab-ac67-4b72-aa0d-b23ddb084cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:05.765680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5482aab-ac67-4b72-aa0d-b23ddb084cb9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:05.768233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 184/354] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:05 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:05.769338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5482aab-ac67-4b72-aa0d-b23ddb084cb9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:05.822921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:05.870483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cc455-fdfd-4796-a3d2-80b2ae6ce2f2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:05.892651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:05.900171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5482aab-ac67-4b72-aa0d-b23ddb084cb9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:05.914185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e216e45-f71a-4224-a3a3-f4f7965857ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:05.915087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e216e45-f71a-4224-a3a3-f4f7965857ce None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.915811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e216e45-f71a-4224-a3a3-f4f7965857ce None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.916105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e216e45-f71a-4224-a3a3-f4f7965857ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.916908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e216e45-f71a-4224-a3a3-f4f7965857ce None admin] RBAC: Adding query filter params (group.id=2aa48136483f469a9c3138ee29808008, role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=395264e10f8345d89e730d4c1a103dd2) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:05.917522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e216e45-f71a-4224-a3a3-f4f7965857ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:05.919005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e216e45-f71a-4224-a3a3-f4f7965857ce None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:05.926823 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 171/355] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:05 2022] GET /identity/v3/role_assignments?group.id=2aa48136483f469a9c3138ee29808008&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:05.928786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6477e4-ef62-4054-a22f-ce00647fa666 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.929327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6477e4-ef62-4054-a22f-ce00647fa666 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.929668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6477e4-ef62-4054-a22f-ce00647fa666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.943403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee79a4e-e65f-4d75-8518-22d806d57382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:05.944140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee79a4e-e65f-4d75-8518-22d806d57382 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:05.944496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee79a4e-e65f-4d75-8518-22d806d57382 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:05.944844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee79a4e-e65f-4d75-8518-22d806d57382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:05.953974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2f6477e4-ef62-4054-a22f-ce00647fa666 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:05.955305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2f6477e4-ef62-4054-a22f-ce00647fa666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:06.038632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee79a4e-e65f-4d75-8518-22d806d57382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:06.041114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee79a4e-e65f-4d75-8518-22d806d57382 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:06.068268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 172/356] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:06.069486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6477e4-ef62-4054-a22f-ce00647fa666 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:06.090384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:06.164390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 185/357] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:06.165330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee79a4e-e65f-4d75-8518-22d806d57382 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:06.220056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6477e4-ef62-4054-a22f-ce00647fa666 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:06.235727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:06.262753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297fa9-1d13-4e93-b6a2-5dc344315349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:06.263410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297fa9-1d13-4e93-b6a2-5dc344315349 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.263694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297fa9-1d13-4e93-b6a2-5dc344315349 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.263971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41297fa9-1d13-4e93-b6a2-5dc344315349 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.275945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41297fa9-1d13-4e93-b6a2-5dc344315349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-41297fa9-1d13-4e93-b6a2-5dc344315349 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:06.281521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:06.282676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 173/358] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:06.283637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297fa9-1d13-4e93-b6a2-5dc344315349 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:06.301010 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee79a4e-e65f-4d75-8518-22d806d57382 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:06.356205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dd9a1-eab7-4236-ba21-a18724a3326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:06.357667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dd9a1-eab7-4236-ba21-a18724a3326e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.358046 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dd9a1-eab7-4236-ba21-a18724a3326e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.358396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dd9a1-eab7-4236-ba21-a18724a3326e None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.360722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dd9a1-eab7-4236-ba21-a18724a3326e None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:06.362363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0dd9a1-eab7-4236-ba21-a18724a3326e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:06.364067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 186/359] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:06.364753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dd9a1-eab7-4236-ba21-a18724a3326e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:06.378363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:06.416783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41297fa9-1d13-4e93-b6a2-5dc344315349 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:06.456300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159156a-5dfa-4329-8970-8776380f4e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:06.456948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159156a-5dfa-4329-8970-8776380f4e46 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.457222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159156a-5dfa-4329-8970-8776380f4e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.457490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159156a-5dfa-4329-8970-8776380f4e46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.470125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159156a-5dfa-4329-8970-8776380f4e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c159156a-5dfa-4329-8970-8776380f4e46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:06.476008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:06.477064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:06.477877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:06.480399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 174/360] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:06.481707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02984e-7280-480a-a991-3764967e7263 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.481984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02984e-7280-480a-a991-3764967e7263 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.482254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02984e-7280-480a-a991-3764967e7263 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.483906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 175/361] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:06.484689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02984e-7280-480a-a991-3764967e7263 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:06.501660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0dd9a1-eab7-4236-ba21-a18724a3326e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:06.531980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:06.551974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c811649-8023-4295-adc2-5136fbc6c501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:06.552620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c811649-8023-4295-adc2-5136fbc6c501 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.552919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c811649-8023-4295-adc2-5136fbc6c501 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.553163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c811649-8023-4295-adc2-5136fbc6c501 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.554318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c811649-8023-4295-adc2-5136fbc6c501 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:06.555831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c811649-8023-4295-adc2-5136fbc6c501 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:06.557606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 187/362] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:06.558376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c811649-8023-4295-adc2-5136fbc6c501 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:06.630111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02984e-7280-480a-a991-3764967e7263 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:06.650767 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:06.673274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acef68-f62e-4042-8b89-7f4df93c53ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:06.674009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acef68-f62e-4042-8b89-7f4df93c53ad None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.674289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acef68-f62e-4042-8b89-7f4df93c53ad None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.674590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acef68-f62e-4042-8b89-7f4df93c53ad None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.675314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acef68-f62e-4042-8b89-7f4df93c53ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:06.675613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acef68-f62e-4042-8b89-7f4df93c53ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:06.676997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acef68-f62e-4042-8b89-7f4df93c53ad None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:06.680912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c811649-8023-4295-adc2-5136fbc6c501 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:06.691496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 176/363] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:06.697093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd4330-c76b-4d8d-a3e0-f720354ef80b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.697497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd4330-c76b-4d8d-a3e0-f720354ef80b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.697864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd4330-c76b-4d8d-a3e0-f720354ef80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.721839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cacd4330-c76b-4d8d-a3e0-f720354ef80b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:06.724279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cacd4330-c76b-4d8d-a3e0-f720354ef80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:06.731941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428d5e3-7e6e-4ac2-9bf0-f535c4f93514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:06.732773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428d5e3-7e6e-4ac2-9bf0-f535c4f93514 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.733415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428d5e3-7e6e-4ac2-9bf0-f535c4f93514 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.733698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428d5e3-7e6e-4ac2-9bf0-f535c4f93514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.735848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428d5e3-7e6e-4ac2-9bf0-f535c4f93514 None admin] RBAC: Adding query filter params (group.id=2aa48136483f469a9c3138ee29808008, role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=395264e10f8345d89e730d4c1a103dd2) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:06.735957 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428d5e3-7e6e-4ac2-9bf0-f535c4f93514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:06.737358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428d5e3-7e6e-4ac2-9bf0-f535c4f93514 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:06.743958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 188/364] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/role_assignments?group.id=2aa48136483f469a9c3138ee29808008&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:06.744663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428d5e3-7e6e-4ac2-9bf0-f535c4f93514 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:06.816511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 177/365] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:06.817904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd4330-c76b-4d8d-a3e0-f720354ef80b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:06.853581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:06.877259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428d5e3-7e6e-4ac2-9bf0-f535c4f93514 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:06.923002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaa88e-e9af-4f47-a6c1-57957a7ea922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:06.923671 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaa88e-e9af-4f47-a6c1-57957a7ea922 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.923956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaa88e-e9af-4f47-a6c1-57957a7ea922 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.924230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaa88e-e9af-4f47-a6c1-57957a7ea922 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.924959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaa88e-e9af-4f47-a6c1-57957a7ea922 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:06.925265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaa88e-e9af-4f47-a6c1-57957a7ea922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:06.926779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaa88e-e9af-4f47-a6c1-57957a7ea922 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:06.940541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 189/366] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:06.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaa88e-e9af-4f47-a6c1-57957a7ea922 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:06.946484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:06.952357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd4330-c76b-4d8d-a3e0-f720354ef80b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:06.992138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8afb4-d35a-41b1-8176-0072901a5a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:06.992787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8afb4-d35a-41b1-8176-0072901a5a79 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:06.993063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8afb4-d35a-41b1-8176-0072901a5a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:06.993340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8afb4-d35a-41b1-8176-0072901a5a79 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:06.994539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8afb4-d35a-41b1-8176-0072901a5a79 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:06.995980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8afb4-d35a-41b1-8176-0072901a5a79 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:06.997744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 178/367] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:07.000686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8afb4-d35a-41b1-8176-0072901a5a79 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:07.077547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaa88e-e9af-4f47-a6c1-57957a7ea922 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:07.124975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d526a-1c99-41aa-bf2f-58b2ba60d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:07.125870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d526a-1c99-41aa-bf2f-58b2ba60d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:07.126294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d526a-1c99-41aa-bf2f-58b2ba60d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:07.126739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d526a-1c99-41aa-bf2f-58b2ba60d8b8 None admin] PATH_INFO: `/v3/users/7176bb3b038e41198f73931b761a98b0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:07.128589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d526a-1c99-41aa-bf2f-58b2ba60d8b8 None admin] RBAC: Authorizing `identity:get_user(user_id=7176bb3b038e41198f73931b761a98b0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:07.130535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d526a-1c99-41aa-bf2f-58b2ba60d8b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:07.131579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:07.133758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 190/368] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/users/7176bb3b038e41198f73931b761a98b0 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:07.135251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d526a-1c99-41aa-bf2f-58b2ba60d8b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:07.153582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8afb4-d35a-41b1-8176-0072901a5a79 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:07.195575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068eb814-4ac0-4d4f-befb-5e72a095d26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:07.196605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eb814-4ac0-4d4f-befb-5e72a095d26f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:07.197051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eb814-4ac0-4d4f-befb-5e72a095d26f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:07.197513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eb814-4ac0-4d4f-befb-5e72a095d26f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:07.211727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068eb814-4ac0-4d4f-befb-5e72a095d26f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-068eb814-4ac0-4d4f-befb-5e72a095d26f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:07.218058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:07.219318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:07.220231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:07.222251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 179/369] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:07.223254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068eb814-4ac0-4d4f-befb-5e72a095d26f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:07.224607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:07.258043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d526a-1c99-41aa-bf2f-58b2ba60d8b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:07.296056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254f29e-eef9-464d-b569-965528b64971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:07.296881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254f29e-eef9-464d-b569-965528b64971 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:07.297000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254f29e-eef9-464d-b569-965528b64971 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:07.297221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254f29e-eef9-464d-b569-965528b64971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:07.340260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068eb814-4ac0-4d4f-befb-5e72a095d26f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:07.373046 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254f29e-eef9-464d-b569-965528b64971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:07.376170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254f29e-eef9-464d-b569-965528b64971 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:07.379414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcba52-6c0a-4282-8b24-59e452d59bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:07.380979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcba52-6c0a-4282-8b24-59e452d59bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:07.381368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcba52-6c0a-4282-8b24-59e452d59bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:07.381751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcba52-6c0a-4282-8b24-59e452d59bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:07.383332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcba52-6c0a-4282-8b24-59e452d59bc3 None admin] RBAC: Adding query filter params (group.id=2aa48136483f469a9c3138ee29808008, role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:07.383332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcba52-6c0a-4282-8b24-59e452d59bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:07.384593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcba52-6c0a-4282-8b24-59e452d59bc3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:07.393124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 180/370] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:07 2022] GET /identity/v3/role_assignments?group.id=2aa48136483f469a9c3138ee29808008&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:07.398462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcba52-6c0a-4282-8b24-59e452d59bc3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:07.421794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:07.534876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:07.595859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 191/371] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:07.598178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254f29e-eef9-464d-b569-965528b64971 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:07.620911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcba52-6c0a-4282-8b24-59e452d59bc3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:07.663126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61405b32-40d1-402c-8b56-f3075a55a309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:07.664104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61405b32-40d1-402c-8b56-f3075a55a309 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:07.664618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61405b32-40d1-402c-8b56-f3075a55a309 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:07.665170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61405b32-40d1-402c-8b56-f3075a55a309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:07.666150 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61405b32-40d1-402c-8b56-f3075a55a309 None admin] RBAC: Adding query filter params (role.id=6cce74d600ba46a2b7786018d0836956, user.id=7176bb3b038e41198f73931b761a98b0, scope.system=all) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:07.666705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61405b32-40d1-402c-8b56-f3075a55a309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:07.668541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61405b32-40d1-402c-8b56-f3075a55a309 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:07.694206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 181/372] 10.222.0.20 () {56 vars in 1308 bytes} [Sat Jul 30 04:45:07 2022] GET /identity/v3/role_assignments?user.id=7176bb3b038e41198f73931b761a98b0&scope.system=all&role.id=6cce74d600ba46a2b7786018d0836956 => generated 510 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:07.699618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61405b32-40d1-402c-8b56-f3075a55a309 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:07.722464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254f29e-eef9-464d-b569-965528b64971 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:07.764860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058d2b-0e77-4378-9418-9507b2644b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:07.765515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058d2b-0e77-4378-9418-9507b2644b3e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:07.765797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058d2b-0e77-4378-9418-9507b2644b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:07.766071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058d2b-0e77-4378-9418-9507b2644b3e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:07.770394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf058d2b-0e77-4378-9418-9507b2644b3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:07.770394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf058d2b-0e77-4378-9418-9507b2644b3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:07.770394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf058d2b-0e77-4378-9418-9507b2644b3e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:07.786546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 192/373] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:07.790367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058d2b-0e77-4378-9418-9507b2644b3e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:07.841077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:07.860699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:07.951850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058d2b-0e77-4378-9418-9507b2644b3e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:07.976450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61405b32-40d1-402c-8b56-f3075a55a309 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:08.000484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708f546-b4df-4305-bc34-86cc347f904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:08.001575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708f546-b4df-4305-bc34-86cc347f904d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:08.001863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708f546-b4df-4305-bc34-86cc347f904d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:08.002144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708f546-b4df-4305-bc34-86cc347f904d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:08.058676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97e32b-3caf-4788-afb5-261f50b902d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:08.059310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97e32b-3caf-4788-afb5-261f50b902d8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:08.059525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97e32b-3caf-4788-afb5-261f50b902d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:08.059792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97e32b-3caf-4788-afb5-261f50b902d8 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:08.062279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97e32b-3caf-4788-afb5-261f50b902d8 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:08.063786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97e32b-3caf-4788-afb5-261f50b902d8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:08.069655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 182/374] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:07 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:08.075420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97e32b-3caf-4788-afb5-261f50b902d8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:08.132144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708f546-b4df-4305-bc34-86cc347f904d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:08.133634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708f546-b4df-4305-bc34-86cc347f904d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:08.188709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:08.227829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97e32b-3caf-4788-afb5-261f50b902d8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:08.293174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ee2fe-f52d-4a98-93cf-b692c47c7be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:08.294180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ee2fe-f52d-4a98-93cf-b692c47c7be9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:08.294683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ee2fe-f52d-4a98-93cf-b692c47c7be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:08.295158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ee2fe-f52d-4a98-93cf-b692c47c7be9 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:08.297259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ee2fe-f52d-4a98-93cf-b692c47c7be9 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:08.298954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ee2fe-f52d-4a98-93cf-b692c47c7be9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:08.303185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 183/375] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:08 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:08.306774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 193/376] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:08.307863 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ee2fe-f52d-4a98-93cf-b692c47c7be9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:08.313061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708f546-b4df-4305-bc34-86cc347f904d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:08.330590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:08.489817 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ee2fe-f52d-4a98-93cf-b692c47c7be9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:08.492119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708f546-b4df-4305-bc34-86cc347f904d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:08.544014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:08.549565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511cadc-8a22-4aa2-9819-a01d4bd2613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:08.550499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511cadc-8a22-4aa2-9819-a01d4bd2613b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:08.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511cadc-8a22-4aa2-9819-a01d4bd2613b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:08.551191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cc11c-4069-4bdc-8884-520c00144182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:08.551672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511cadc-8a22-4aa2-9819-a01d4bd2613b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:08.551840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cc11c-4069-4bdc-8884-520c00144182 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:08.552124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cc11c-4069-4bdc-8884-520c00144182 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:08.552401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cc11c-4069-4bdc-8884-520c00144182 None admin] PATH_INFO: `/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:08.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511cadc-8a22-4aa2-9819-a01d4bd2613b None admin] RBAC: Adding query filter params (group.id=2aa48136483f469a9c3138ee29808008, role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:08.553609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cc11c-4069-4bdc-8884-520c00144182 None admin] RBAC: Authorizing `identity:get_project(project_id=2e75ff1f24d24ead87dfc7d32ddc1723)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:08.553947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511cadc-8a22-4aa2-9819-a01d4bd2613b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:08.555891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511cadc-8a22-4aa2-9819-a01d4bd2613b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:08.556121 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cc11c-4069-4bdc-8884-520c00144182 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:08.558335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 194/377] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:08 2022] GET /identity/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723 => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:08.560133 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9642a-ecfd-48d2-87dc-20d48c7107ac None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:08.560428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9642a-ecfd-48d2-87dc-20d48c7107ac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:08.560709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9642a-ecfd-48d2-87dc-20d48c7107ac None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:08.563849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 184/379] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:08 2022] GET /identity/v3/role_assignments?group.id=2aa48136483f469a9c3138ee29808008&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:08.564894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511cadc-8a22-4aa2-9819-a01d4bd2613b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:08.566647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 195/379] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:08.571920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d583a71-b374-4f5a-85da-b3023aacebf8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:08.572248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d583a71-b374-4f5a-85da-b3023aacebf8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:08.572491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d583a71-b374-4f5a-85da-b3023aacebf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:08.615695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3d583a71-b374-4f5a-85da-b3023aacebf8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:08.616935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3d583a71-b374-4f5a-85da-b3023aacebf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:08.748276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:08.755865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 196/380] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:08.783039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511cadc-8a22-4aa2-9819-a01d4bd2613b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:08.804870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d583a71-b374-4f5a-85da-b3023aacebf8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:08.837110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc59bf-145a-4cb8-8bcf-6be185f4fe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:08.838103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc59bf-145a-4cb8-8bcf-6be185f4fe95 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:08.839485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc59bf-145a-4cb8-8bcf-6be185f4fe95 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:08.839960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc59bf-145a-4cb8-8bcf-6be185f4fe95 None admin] PATH_INFO: `/v3/groups/f2106ba5551149e4af3daf5c0406e263` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:08.852886 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc59bf-145a-4cb8-8bcf-6be185f4fe95 None admin] RBAC: Authorizing `identity:get_group(group_id=f2106ba5551149e4af3daf5c0406e263)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:08.856320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc59bf-145a-4cb8-8bcf-6be185f4fe95 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:08.860853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 185/381] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:08 2022] GET /identity/v3/groups/f2106ba5551149e4af3daf5c0406e263 => generated 229 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:08.864384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc59bf-145a-4cb8-8bcf-6be185f4fe95 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:09.020336 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d583a71-b374-4f5a-85da-b3023aacebf8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:09.073746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:09.082918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4400f-0881-45e1-aee3-95a7382e8631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:09.083578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4400f-0881-45e1-aee3-95a7382e8631 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:09.083856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4400f-0881-45e1-aee3-95a7382e8631 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:09.084125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4400f-0881-45e1-aee3-95a7382e8631 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:09.094471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc59bf-145a-4cb8-8bcf-6be185f4fe95 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:09.098523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4400f-0881-45e1-aee3-95a7382e8631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0eb4400f-0881-45e1-aee3-95a7382e8631 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:09.105108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:09.106012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:09.106853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 197/382] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:09.114076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4400f-0881-45e1-aee3-95a7382e8631 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:09.197588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447436d6-4cbc-4984-9cd0-f9a4b2a8690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:09.198237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447436d6-4cbc-4984-9cd0-f9a4b2a8690e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:09.198558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447436d6-4cbc-4984-9cd0-f9a4b2a8690e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:09.198837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447436d6-4cbc-4984-9cd0-f9a4b2a8690e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:09.199649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447436d6-4cbc-4984-9cd0-f9a4b2a8690e None admin] RBAC: Adding query filter params (group.id=f2106ba5551149e4af3daf5c0406e263, role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=2e75ff1f24d24ead87dfc7d32ddc1723) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:09.199959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447436d6-4cbc-4984-9cd0-f9a4b2a8690e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:09.201407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447436d6-4cbc-4984-9cd0-f9a4b2a8690e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:09.208859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 186/383] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:08 2022] GET /identity/v3/role_assignments?group.id=f2106ba5551149e4af3daf5c0406e263&scope.project.id=2e75ff1f24d24ead87dfc7d32ddc1723&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 260 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:09.210821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:09.247705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4400f-0881-45e1-aee3-95a7382e8631 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:09.296635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0175ad-ca99-4f46-9219-aba2213786a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:09.297283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0175ad-ca99-4f46-9219-aba2213786a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:09.297569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0175ad-ca99-4f46-9219-aba2213786a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:09.297850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0175ad-ca99-4f46-9219-aba2213786a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:09.298626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0175ad-ca99-4f46-9219-aba2213786a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:09.298937 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0175ad-ca99-4f46-9219-aba2213786a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:09.300363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0175ad-ca99-4f46-9219-aba2213786a7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:09.314597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 198/384] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:45:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:09.316267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447436d6-4cbc-4984-9cd0-f9a4b2a8690e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:09.372481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958393b-6a65-45b3-8e5e-528f8b1502e3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:09.372777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958393b-6a65-45b3-8e5e-528f8b1502e3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:09.372958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958393b-6a65-45b3-8e5e-528f8b1502e3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:09.374852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 199/385] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:45:09.389942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c98f93-6744-4598-b868-2ec08b95c622 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:09.390243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c98f93-6744-4598-b868-2ec08b95c622 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:09.390519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c98f93-6744-4598-b868-2ec08b95c622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:09.422219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-71c98f93-6744-4598-b868-2ec08b95c622 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:09.423468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71c98f93-6744-4598-b868-2ec08b95c622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:09.457832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447436d6-4cbc-4984-9cd0-f9a4b2a8690e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:09.513034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0b65-acad-4f56-b735-ea31eb847812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:09.513694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0b65-acad-4f56-b735-ea31eb847812 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:09.513973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0b65-acad-4f56-b735-ea31eb847812 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:09.514331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0b65-acad-4f56-b735-ea31eb847812 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:09.516104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd0b65-acad-4f56-b735-ea31eb847812 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:09.517553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd0b65-acad-4f56-b735-ea31eb847812 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:09.519391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 187/386] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:09 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:09.525905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0b65-acad-4f56-b735-ea31eb847812 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:09.526778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:09.565160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 200/387] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:09.593890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c98f93-6744-4598-b868-2ec08b95c622 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:09.642885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:09.701833 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0b65-acad-4f56-b735-ea31eb847812 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:09.766954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f29c-f5d8-4454-96fc-38dbcb517c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:09.768192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f29c-f5d8-4454-96fc-38dbcb517c6e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:09.768684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f29c-f5d8-4454-96fc-38dbcb517c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:09.768959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f29c-f5d8-4454-96fc-38dbcb517c6e None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:09.771293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c98f93-6744-4598-b868-2ec08b95c622 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:09.773210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f29c-f5d8-4454-96fc-38dbcb517c6e None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:09.774694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f29c-f5d8-4454-96fc-38dbcb517c6e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:09.778020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 188/388] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:09 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:09.787449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f29c-f5d8-4454-96fc-38dbcb517c6e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:09.838888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb504b5-5ca4-4748-bfca-6722a1d78dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:09.839843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb504b5-5ca4-4748-bfca-6722a1d78dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:09.840292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb504b5-5ca4-4748-bfca-6722a1d78dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:09.840757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb504b5-5ca4-4748-bfca-6722a1d78dd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:09.865055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb504b5-5ca4-4748-bfca-6722a1d78dd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dfb504b5-5ca4-4748-bfca-6722a1d78dd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:09.874836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:09.875913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:09.876919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:09.881319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 201/389] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:09.886486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb504b5-5ca4-4748-bfca-6722a1d78dd2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:09.953321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:09.957316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f29c-f5d8-4454-96fc-38dbcb517c6e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:10.001911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d79c0-8d7f-43f1-8da0-5e6e3e923106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:10.011616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d79c0-8d7f-43f1-8da0-5e6e3e923106 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:10.012033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d79c0-8d7f-43f1-8da0-5e6e3e923106 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:10.012486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d79c0-8d7f-43f1-8da0-5e6e3e923106 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2/groups/2aa48136483f469a9c3138ee29808008/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:10.015471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d79c0-8d7f-43f1-8da0-5e6e3e923106 None admin] RBAC: Authorizing `identity:create_grant(project_id=395264e10f8345d89e730d4c1a103dd2, group_id=2aa48136483f469a9c3138ee29808008, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:10.022895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d79c0-8d7f-43f1-8da0-5e6e3e923106 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:10.027366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:10.042821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 189/390] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Jul 30 04:45:09 2022] PUT /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2/groups/2aa48136483f469a9c3138ee29808008/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:10.070478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9bbca-2d92-4c3b-864e-1dac545fc513 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:10.070478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9bbca-2d92-4c3b-864e-1dac545fc513 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:10.070478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9bbca-2d92-4c3b-864e-1dac545fc513 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:10.070478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 190/391] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:45:10.080263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781840-489f-49cd-8098-010c81fc7a53 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:10.080709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781840-489f-49cd-8098-010c81fc7a53 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:10.080988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781840-489f-49cd-8098-010c81fc7a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:10.099945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb504b5-5ca4-4748-bfca-6722a1d78dd2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:10.112689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7d781840-489f-49cd-8098-010c81fc7a53 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:10.113911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7d781840-489f-49cd-8098-010c81fc7a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:10.206535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 191/392] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:10.229520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57911435-a875-4c1c-993c-db986d02a921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:10.230165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57911435-a875-4c1c-993c-db986d02a921 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:10.230465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57911435-a875-4c1c-993c-db986d02a921 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:10.230744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57911435-a875-4c1c-993c-db986d02a921 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:10.231483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57911435-a875-4c1c-993c-db986d02a921 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:10.231790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57911435-a875-4c1c-993c-db986d02a921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:10.233275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57911435-a875-4c1c-993c-db986d02a921 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:10.239620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d781840-489f-49cd-8098-010c81fc7a53 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:10.245535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 202/393] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:10.252562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57911435-a875-4c1c-993c-db986d02a921 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:10.367731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d781840-489f-49cd-8098-010c81fc7a53 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:10.385328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57911435-a875-4c1c-993c-db986d02a921 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:10.394929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:10.421263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:10.431708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632b560-84b5-48ba-9422-cc1431e25a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:10.432352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632b560-84b5-48ba-9422-cc1431e25a09 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:10.433327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632b560-84b5-48ba-9422-cc1431e25a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:10.433605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632b560-84b5-48ba-9422-cc1431e25a09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:10.450525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc246a38-991d-416d-9c15-fa52b9e3ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:10.451166 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc246a38-991d-416d-9c15-fa52b9e3ae71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:10.451437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc246a38-991d-416d-9c15-fa52b9e3ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:10.452378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc246a38-991d-416d-9c15-fa52b9e3ae71 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:10.456363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc246a38-991d-416d-9c15-fa52b9e3ae71 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:10.457530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc246a38-991d-416d-9c15-fa52b9e3ae71 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:10.459753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632b560-84b5-48ba-9422-cc1431e25a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a632b560-84b5-48ba-9422-cc1431e25a09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:10.469324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:10.470280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:10.471191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:10.471851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 203/394] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:10 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:10.471851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc246a38-991d-416d-9c15-fa52b9e3ae71 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:10.476725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 192/395] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:10.480478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632b560-84b5-48ba-9422-cc1431e25a09 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:10.660635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc246a38-991d-416d-9c15-fa52b9e3ae71 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:10.664861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632b560-84b5-48ba-9422-cc1431e25a09 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:10.727182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682eeca-065a-4904-a37d-0d3712d3ec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:10.727755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682eeca-065a-4904-a37d-0d3712d3ec6f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:10.728033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682eeca-065a-4904-a37d-0d3712d3ec6f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:10.728307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682eeca-065a-4904-a37d-0d3712d3ec6f None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:10.729508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32431420-e59b-43be-a827-0a94b6c2df7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:10.730125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32431420-e59b-43be-a827-0a94b6c2df7f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:10.730397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32431420-e59b-43be-a827-0a94b6c2df7f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:10.730688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32431420-e59b-43be-a827-0a94b6c2df7f None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:10.731408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32431420-e59b-43be-a827-0a94b6c2df7f None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:10.731705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32431420-e59b-43be-a827-0a94b6c2df7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:10.733699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32431420-e59b-43be-a827-0a94b6c2df7f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:10.736000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682eeca-065a-4904-a37d-0d3712d3ec6f None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:10.737441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682eeca-065a-4904-a37d-0d3712d3ec6f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:10.744003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 204/396] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:10 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:10.746082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682eeca-065a-4904-a37d-0d3712d3ec6f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:10.760418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 193/397] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:45:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:10.765974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32431420-e59b-43be-a827-0a94b6c2df7f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:10.817057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:10.833910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:10.903678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32431420-e59b-43be-a827-0a94b6c2df7f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:10.960204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682eeca-065a-4904-a37d-0d3712d3ec6f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:10.979023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db21bc-85b1-46bc-ac20-38bcd3250a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:10.980240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db21bc-85b1-46bc-ac20-38bcd3250a78 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:10.980684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db21bc-85b1-46bc-ac20-38bcd3250a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:10.981107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db21bc-85b1-46bc-ac20-38bcd3250a78 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:10.982814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db21bc-85b1-46bc-ac20-38bcd3250a78 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:10.986182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db21bc-85b1-46bc-ac20-38bcd3250a78 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:10.991386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 194/398] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:10 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:10.994816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db21bc-85b1-46bc-ac20-38bcd3250a78 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:11.057702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b4667-7b29-49bd-b512-afebbd6aca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:11.058665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b4667-7b29-49bd-b512-afebbd6aca51 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.059118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b4667-7b29-49bd-b512-afebbd6aca51 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.059554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b4667-7b29-49bd-b512-afebbd6aca51 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2/groups/2aa48136483f469a9c3138ee29808008/roles/9ec4568f149c40c38cd07fb317a40280` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.064150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b4667-7b29-49bd-b512-afebbd6aca51 None admin] RBAC: Authorizing `identity:create_grant(project_id=395264e10f8345d89e730d4c1a103dd2, group_id=2aa48136483f469a9c3138ee29808008, role_id=9ec4568f149c40c38cd07fb317a40280)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:11.065783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b4667-7b29-49bd-b512-afebbd6aca51 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:11.100539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 205/399] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Jul 30 04:45:10 2022] PUT /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2/groups/2aa48136483f469a9c3138ee29808008/roles/9ec4568f149c40c38cd07fb317a40280 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:45:11.150670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed9ee4-dbd4-4568-b7ca-824c92b9c9ea None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.150962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed9ee4-dbd4-4568-b7ca-824c92b9c9ea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.151247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed9ee4-dbd4-4568-b7ca-824c92b9c9ea None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.153193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 206/400] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:45:11.157529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db21bc-85b1-46bc-ac20-38bcd3250a78 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:11.158970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8d00-3df0-422a-aa6f-419c9bfb176e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.159258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8d00-3df0-422a-aa6f-419c9bfb176e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.159538 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab8d00-3df0-422a-aa6f-419c9bfb176e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.178928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:11.181745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f2ab8d00-3df0-422a-aa6f-419c9bfb176e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:11.183052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f2ab8d00-3df0-422a-aa6f-419c9bfb176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:11.205676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b072d-7bbb-497d-af1b-3faba6cfb4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:11.206715 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b072d-7bbb-497d-af1b-3faba6cfb4dd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.207140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b072d-7bbb-497d-af1b-3faba6cfb4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.207555 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b072d-7bbb-497d-af1b-3faba6cfb4dd None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.209616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b072d-7bbb-497d-af1b-3faba6cfb4dd None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:11.211645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b072d-7bbb-497d-af1b-3faba6cfb4dd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:11.215138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 195/401] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:10 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:11.222454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b072d-7bbb-497d-af1b-3faba6cfb4dd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:11.288112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 207/402] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:11.355723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab8d00-3df0-422a-aa6f-419c9bfb176e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:11.388508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b072d-7bbb-497d-af1b-3faba6cfb4dd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:11.394757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:11.431959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1102ed-ab9c-4a76-9d52-09351edba100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:11.432794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1102ed-ab9c-4a76-9d52-09351edba100 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.433066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1102ed-ab9c-4a76-9d52-09351edba100 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.433349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1102ed-ab9c-4a76-9d52-09351edba100 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/groups/2aa48136483f469a9c3138ee29808008/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.435450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1102ed-ab9c-4a76-9d52-09351edba100 None admin] RBAC: Authorizing `identity:create_grant(project_id=733b6ca2bdf14132a5de09c5371d80ae, group_id=2aa48136483f469a9c3138ee29808008, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:11.437067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1102ed-ab9c-4a76-9d52-09351edba100 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:11.454238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 196/403] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Jul 30 04:45:11 2022] PUT /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/groups/2aa48136483f469a9c3138ee29808008/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:45:11.488979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab8d00-3df0-422a-aa6f-419c9bfb176e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:11.530950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67a15f-3eba-4923-8f67-c4de692243fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:11.532020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67a15f-3eba-4923-8f67-c4de692243fc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.532447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67a15f-3eba-4923-8f67-c4de692243fc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.532872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67a15f-3eba-4923-8f67-c4de692243fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.545453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67a15f-3eba-4923-8f67-c4de692243fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8f67a15f-3eba-4923-8f67-c4de692243fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:11.550982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:11.552153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:11.553229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:11.555669 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 208/404] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:11.561994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1102ed-ab9c-4a76-9d52-09351edba100 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:11.617138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:11.675385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36f62f-0174-478c-9d59-32065a632e96 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.675664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36f62f-0174-478c-9d59-32065a632e96 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.675942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36f62f-0174-478c-9d59-32065a632e96 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.677392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 209/405] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:45:11.681124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1102ed-ab9c-4a76-9d52-09351edba100 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:11.682873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c820e5-2f81-4223-8af7-e772c97e8d87 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.683159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c820e5-2f81-4223-8af7-e772c97e8d87 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.683437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c820e5-2f81-4223-8af7-e772c97e8d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.704948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b1c820e5-2f81-4223-8af7-e772c97e8d87 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:11.706154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b1c820e5-2f81-4223-8af7-e772c97e8d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:11.727477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe19d9-1397-431c-a9ff-0dca46bfb7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:11.728107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe19d9-1397-431c-a9ff-0dca46bfb7fc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.728383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe19d9-1397-431c-a9ff-0dca46bfb7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.728654 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe19d9-1397-431c-a9ff-0dca46bfb7fc None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.729379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe19d9-1397-431c-a9ff-0dca46bfb7fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:11.731246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe19d9-1397-431c-a9ff-0dca46bfb7fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:11.732652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe19d9-1397-431c-a9ff-0dca46bfb7fc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:11.752526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 197/406] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:11.758278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe19d9-1397-431c-a9ff-0dca46bfb7fc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:11.840106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 210/407] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:11.879955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe19d9-1397-431c-a9ff-0dca46bfb7fc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:11.892092 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:11.895722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c820e5-2f81-4223-8af7-e772c97e8d87 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:11.927944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cfe98-cbcf-4404-9372-119698053e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:11.928651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cfe98-cbcf-4404-9372-119698053e48 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:11.928934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cfe98-cbcf-4404-9372-119698053e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:11.929205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cfe98-cbcf-4404-9372-119698053e48 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:11.930522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cfe98-cbcf-4404-9372-119698053e48 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:11.931977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cfe98-cbcf-4404-9372-119698053e48 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:11.933573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 198/408] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:11 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:11.939325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cfe98-cbcf-4404-9372-119698053e48 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:11.962823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:12.093988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c820e5-2f81-4223-8af7-e772c97e8d87 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:12.121531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cfe98-cbcf-4404-9372-119698053e48 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:12.151169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336a00-a8f8-4eb6-83a1-e89c8e11f341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:12.152105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336a00-a8f8-4eb6-83a1-e89c8e11f341 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:12.152372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336a00-a8f8-4eb6-83a1-e89c8e11f341 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:12.152651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336a00-a8f8-4eb6-83a1-e89c8e11f341 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:12.159550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d97a50-7985-42a4-8eab-f6cc5dc52860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:12.160620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d97a50-7985-42a4-8eab-f6cc5dc52860 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:12.161109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d97a50-7985-42a4-8eab-f6cc5dc52860 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:12.161701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d97a50-7985-42a4-8eab-f6cc5dc52860 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:12.163196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d97a50-7985-42a4-8eab-f6cc5dc52860 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:12.164978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d97a50-7985-42a4-8eab-f6cc5dc52860 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:12.167244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 199/409] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:11 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:12.173237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d97a50-7985-42a4-8eab-f6cc5dc52860 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:12.188757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94336a00-a8f8-4eb6-83a1-e89c8e11f341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-94336a00-a8f8-4eb6-83a1-e89c8e11f341 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:12.194341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:12.195271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:12.196083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:12.202928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 211/410] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:12.209840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336a00-a8f8-4eb6-83a1-e89c8e11f341 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:12.254918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:12.290325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d97a50-7985-42a4-8eab-f6cc5dc52860 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:12.350942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e0322-a3d8-4f71-b18a-a051c250d469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:12.351541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e0322-a3d8-4f71-b18a-a051c250d469 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:12.351823 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e0322-a3d8-4f71-b18a-a051c250d469 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:12.352101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e0322-a3d8-4f71-b18a-a051c250d469 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/groups/2aa48136483f469a9c3138ee29808008/roles/9ec4568f149c40c38cd07fb317a40280` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:12.355883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e0322-a3d8-4f71-b18a-a051c250d469 None admin] RBAC: Authorizing `identity:create_grant(project_id=733b6ca2bdf14132a5de09c5371d80ae, group_id=2aa48136483f469a9c3138ee29808008, role_id=9ec4568f149c40c38cd07fb317a40280)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:12.357334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e0322-a3d8-4f71-b18a-a051c250d469 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:12.385630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 200/411] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Jul 30 04:45:12 2022] PUT /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae/groups/2aa48136483f469a9c3138ee29808008/roles/9ec4568f149c40c38cd07fb317a40280 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:45:12.391263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:12.414337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336a00-a8f8-4eb6-83a1-e89c8e11f341 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:12.467205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9f4e7-ad54-4a3d-b5de-ac983c2d0359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:12.467899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9f4e7-ad54-4a3d-b5de-ac983c2d0359 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:12.468212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9f4e7-ad54-4a3d-b5de-ac983c2d0359 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:12.468568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9f4e7-ad54-4a3d-b5de-ac983c2d0359 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:12.469332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9f4e7-ad54-4a3d-b5de-ac983c2d0359 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:12.469642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9f4e7-ad54-4a3d-b5de-ac983c2d0359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:12.471109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9f4e7-ad54-4a3d-b5de-ac983c2d0359 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:12.487865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 212/412] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:12.493672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e0322-a3d8-4f71-b18a-a051c250d469 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:12.626080 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e0322-a3d8-4f71-b18a-a051c250d469 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:12.627592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:12.666970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57ba09-5873-4370-b08c-8f0ab1814471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:12.667620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57ba09-5873-4370-b08c-8f0ab1814471 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:12.667899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57ba09-5873-4370-b08c-8f0ab1814471 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:12.668177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57ba09-5873-4370-b08c-8f0ab1814471 None admin] PATH_INFO: `/v3/groups/f2106ba5551149e4af3daf5c0406e263` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:12.669339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57ba09-5873-4370-b08c-8f0ab1814471 None admin] RBAC: Authorizing `identity:get_group(group_id=f2106ba5551149e4af3daf5c0406e263)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:12.671634 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57ba09-5873-4370-b08c-8f0ab1814471 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:12.673575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 201/413] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:12 2022] GET /identity/v3/groups/f2106ba5551149e4af3daf5c0406e263 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:12.680097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9f4e7-ad54-4a3d-b5de-ac983c2d0359 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:12.789537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6cb4-0a23-45ba-9806-b8c1b5aa7485 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:12.790033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6cb4-0a23-45ba-9806-b8c1b5aa7485 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:12.790612 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6cb4-0a23-45ba-9806-b8c1b5aa7485 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:12.792548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 202/414] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:12.799584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c6a9c-59a4-4d36-846d-8031f52562cd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:12.800155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c6a9c-59a4-4d36-846d-8031f52562cd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:12.800657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c6a9c-59a4-4d36-846d-8031f52562cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:12.810870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9f4e7-ad54-4a3d-b5de-ac983c2d0359 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:12.823378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c15c6a9c-59a4-4d36-846d-8031f52562cd None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:12.825322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c15c6a9c-59a4-4d36-846d-8031f52562cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:12.891818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba30d25-f9a5-4f65-bbd8-d8d515483fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:12.892447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba30d25-f9a5-4f65-bbd8-d8d515483fdc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:12.892674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba30d25-f9a5-4f65-bbd8-d8d515483fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:12.892951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba30d25-f9a5-4f65-bbd8-d8d515483fdc None admin] PATH_INFO: `/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:12.894141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba30d25-f9a5-4f65-bbd8-d8d515483fdc None admin] RBAC: Authorizing `identity:get_project(project_id=2e75ff1f24d24ead87dfc7d32ddc1723)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:12.895811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba30d25-f9a5-4f65-bbd8-d8d515483fdc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:12.897593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 213/415] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:12 2022] GET /identity/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723 => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:12.903149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba30d25-f9a5-4f65-bbd8-d8d515483fdc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:12.908409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 203/416] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:12.930522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:12.936907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c6a9c-59a4-4d36-846d-8031f52562cd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:13.067531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:13.071728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c6a9c-59a4-4d36-846d-8031f52562cd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:13.111063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f2b7d-5bd0-4e59-a654-f21b2e005ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:13.112114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f2b7d-5bd0-4e59-a654-f21b2e005ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:13.112718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f2b7d-5bd0-4e59-a654-f21b2e005ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:13.113211 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f2b7d-5bd0-4e59-a654-f21b2e005ae4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:13.122471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba30d25-f9a5-4f65-bbd8-d8d515483fdc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:13.125688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f2b7d-5bd0-4e59-a654-f21b2e005ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5e4f2b7d-5bd0-4e59-a654-f21b2e005ae4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:13.131515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:13.132424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:13.133219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:13.137104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 204/417] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:13.143944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f2b7d-5bd0-4e59-a654-f21b2e005ae4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:13.216065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb742610-aa28-465e-a8f9-971e7c7f8ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:13.216762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb742610-aa28-465e-a8f9-971e7c7f8ce3 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:13.217052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb742610-aa28-465e-a8f9-971e7c7f8ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:13.217889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb742610-aa28-465e-a8f9-971e7c7f8ce3 None admin] PATH_INFO: `/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723/groups/f2106ba5551149e4af3daf5c0406e263/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:13.223820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb742610-aa28-465e-a8f9-971e7c7f8ce3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e75ff1f24d24ead87dfc7d32ddc1723, group_id=f2106ba5551149e4af3daf5c0406e263, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:13.225293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb742610-aa28-465e-a8f9-971e7c7f8ce3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:13.261996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f2b7d-5bd0-4e59-a654-f21b2e005ae4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:13.272981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 214/418] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Jul 30 04:45:12 2022] PUT /identity/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723/groups/f2106ba5551149e4af3daf5c0406e263/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:13.303965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3637d9-3364-41d9-8caa-9cd51037c05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:13.305014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3637d9-3364-41d9-8caa-9cd51037c05b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:13.305500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3637d9-3364-41d9-8caa-9cd51037c05b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:13.306022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3637d9-3364-41d9-8caa-9cd51037c05b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:13.307010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3637d9-3364-41d9-8caa-9cd51037c05b None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:13.307558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3637d9-3364-41d9-8caa-9cd51037c05b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:13.309244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3637d9-3364-41d9-8caa-9cd51037c05b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:13.320320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 205/419] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:45:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:13.327286 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb742610-aa28-465e-a8f9-971e7c7f8ce3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:13.450984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:13.486130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb742610-aa28-465e-a8f9-971e7c7f8ce3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:13.522923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8ca27-ec1d-4ca0-a788-9426dd407d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:13.523611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8ca27-ec1d-4ca0-a788-9426dd407d4d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:13.523897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8ca27-ec1d-4ca0-a788-9426dd407d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:13.524173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8ca27-ec1d-4ca0-a788-9426dd407d4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:13.606658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8ca27-ec1d-4ca0-a788-9426dd407d4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:13.608753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8ca27-ec1d-4ca0-a788-9426dd407d4d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:13.705933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:13.718936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add32bc5-7321-4741-b4be-59088dd96de6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:13.719796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add32bc5-7321-4741-b4be-59088dd96de6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:13.720291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add32bc5-7321-4741-b4be-59088dd96de6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:13.722095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 206/420] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:13.727447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736da8d2-c0ce-47d6-9561-f0dc41b7d1c6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:13.727977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736da8d2-c0ce-47d6-9561-f0dc41b7d1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:13.728456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736da8d2-c0ce-47d6-9561-f0dc41b7d1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:13.740397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 215/421] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:13.748437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8ca27-ec1d-4ca0-a788-9426dd407d4d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:13.754341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-736da8d2-c0ce-47d6-9561-f0dc41b7d1c6 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:13.755932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-736da8d2-c0ce-47d6-9561-f0dc41b7d1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:13.791343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:13.855764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 207/422] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:13.857144 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2977b7-e055-4ec2-9073-39011cc64c9a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:13.857429 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2977b7-e055-4ec2-9073-39011cc64c9a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:13.857700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2977b7-e055-4ec2-9073-39011cc64c9a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:13.860586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 208/423] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:13.865837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8200d8-aa6d-4433-86eb-cfda52c40597 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:13.866125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8200d8-aa6d-4433-86eb-cfda52c40597 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:13.866484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8200d8-aa6d-4433-86eb-cfda52c40597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:13.886680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ce8200d8-aa6d-4433-86eb-cfda52c40597 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:13.887991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ce8200d8-aa6d-4433-86eb-cfda52c40597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:13.894806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8ca27-ec1d-4ca0-a788-9426dd407d4d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:13.940111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b658d5d-8b69-4527-9896-38e5cca8f21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:13.940761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b658d5d-8b69-4527-9896-38e5cca8f21d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:13.941042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b658d5d-8b69-4527-9896-38e5cca8f21d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:13.941319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b658d5d-8b69-4527-9896-38e5cca8f21d None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:13.942545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b658d5d-8b69-4527-9896-38e5cca8f21d None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:13.943994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b658d5d-8b69-4527-9896-38e5cca8f21d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:13.945977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 216/424] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:13 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:13.946839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b658d5d-8b69-4527-9896-38e5cca8f21d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:13.972095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 209/425] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:13.973091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8200d8-aa6d-4433-86eb-cfda52c40597 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:14.009505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:14.084410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b658d5d-8b69-4527-9896-38e5cca8f21d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:14.087169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:14.110075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8200d8-aa6d-4433-86eb-cfda52c40597 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:14.141799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16dbdd-a6c3-4f12-a1f8-972c5e23a0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:14.143413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16dbdd-a6c3-4f12-a1f8-972c5e23a0cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:14.144014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16dbdd-a6c3-4f12-a1f8-972c5e23a0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:14.144481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16dbdd-a6c3-4f12-a1f8-972c5e23a0cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:14.151112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eec4dd-6de2-4739-936d-6c2e60cb06a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:14.152102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eec4dd-6de2-4739-936d-6c2e60cb06a5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:14.152450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eec4dd-6de2-4739-936d-6c2e60cb06a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:14.152829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eec4dd-6de2-4739-936d-6c2e60cb06a5 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:14.154354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eec4dd-6de2-4739-936d-6c2e60cb06a5 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:14.156134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eec4dd-6de2-4739-936d-6c2e60cb06a5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:14.158565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16dbdd-a6c3-4f12-a1f8-972c5e23a0cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:14.162732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 210/426] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:13 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:14.163627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eec4dd-6de2-4739-936d-6c2e60cb06a5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9f16dbdd-a6c3-4f12-a1f8-972c5e23a0cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:14.164189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:14.165053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:14.165855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:14.168381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 217/427] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:14.170460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16dbdd-a6c3-4f12-a1f8-972c5e23a0cc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:14.285065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:14.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16dbdd-a6c3-4f12-a1f8-972c5e23a0cc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:14.298485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eec4dd-6de2-4739-936d-6c2e60cb06a5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:14.343304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078c92-8953-44f9-82f6-8067ce3fcbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:14.344060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078c92-8953-44f9-82f6-8067ce3fcbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:14.344289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313da9ca-7dc8-4e61-8bee-62e4e165f395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:14.344551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078c92-8953-44f9-82f6-8067ce3fcbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:14.344930 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70078c92-8953-44f9-82f6-8067ce3fcbd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:14.345395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313da9ca-7dc8-4e61-8bee-62e4e165f395 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:14.345829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313da9ca-7dc8-4e61-8bee-62e4e165f395 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:14.346082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078c92-8953-44f9-82f6-8067ce3fcbd3 None admin] RBAC: Adding query filter params (group.id=2aa48136483f469a9c3138ee29808008, role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=395264e10f8345d89e730d4c1a103dd2) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:14.346405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313da9ca-7dc8-4e61-8bee-62e4e165f395 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:14.346630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078c92-8953-44f9-82f6-8067ce3fcbd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:14.348337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70078c92-8953-44f9-82f6-8067ce3fcbd3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:14.359443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313da9ca-7dc8-4e61-8bee-62e4e165f395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:14.359939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 218/428] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:14 2022] GET /identity/v3/role_assignments?group.id=2aa48136483f469a9c3138ee29808008&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:14.360933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078c92-8953-44f9-82f6-8067ce3fcbd3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-313da9ca-7dc8-4e61-8bee-62e4e165f395 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:14.365296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:14.366428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 30 04:45:14.367474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:14.369694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 211/429] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:14.378353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313da9ca-7dc8-4e61-8bee-62e4e165f395 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:14.402589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:14.576199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70078c92-8953-44f9-82f6-8067ce3fcbd3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:14.600664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313da9ca-7dc8-4e61-8bee-62e4e165f395 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:14.624448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b49000-2d68-480a-94de-5e723cfc6c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:14.625732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b49000-2d68-480a-94de-5e723cfc6c09 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:14.626017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b49000-2d68-480a-94de-5e723cfc6c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:14.626299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b49000-2d68-480a-94de-5e723cfc6c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:14.627095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b49000-2d68-480a-94de-5e723cfc6c09 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:14.627402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b49000-2d68-480a-94de-5e723cfc6c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:14.629716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b49000-2d68-480a-94de-5e723cfc6c09 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:14.644504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 219/430] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:14.649726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1f6e2-6435-45e6-aa4c-a721048373f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:14.650018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b49000-2d68-480a-94de-5e723cfc6c09 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:14.650361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1f6e2-6435-45e6-aa4c-a721048373f2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:14.650683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1f6e2-6435-45e6-aa4c-a721048373f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:14.650958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1f6e2-6435-45e6-aa4c-a721048373f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:14.651952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1f6e2-6435-45e6-aa4c-a721048373f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:14.652253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1f6e2-6435-45e6-aa4c-a721048373f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:14.653731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1f6e2-6435-45e6-aa4c-a721048373f2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:14.663254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 212/431] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:45:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:14.668870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1f6e2-6435-45e6-aa4c-a721048373f2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:14.684575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:14.769934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:14.795236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b49000-2d68-480a-94de-5e723cfc6c09 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:14.815231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1f6e2-6435-45e6-aa4c-a721048373f2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:14.843782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cce97-1509-4ad1-aa75-c1c50bf182ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:14.843782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cce97-1509-4ad1-aa75-c1c50bf182ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:14.844042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cce97-1509-4ad1-aa75-c1c50bf182ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:14.844112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cce97-1509-4ad1-aa75-c1c50bf182ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:14.863198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069763a-a63f-4151-8538-bd0fba74eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:14.864172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069763a-a63f-4151-8538-bd0fba74eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:14.864922 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069763a-a63f-4151-8538-bd0fba74eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:14.865360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069763a-a63f-4151-8538-bd0fba74eaf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:14.931098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cce97-1509-4ad1-aa75-c1c50bf182ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:14.933271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cce97-1509-4ad1-aa75-c1c50bf182ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:14.963421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:14.964071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069763a-a63f-4151-8538-bd0fba74eaf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:14.968347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069763a-a63f-4151-8538-bd0fba74eaf1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:15.078623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 220/432] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:45:15.082482 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cce97-1509-4ad1-aa75-c1c50bf182ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:15.087106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:15.131786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 213/433] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:45:15.137136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069763a-a63f-4151-8538-bd0fba74eaf1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:15.217203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cce97-1509-4ad1-aa75-c1c50bf182ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:15.244908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:15.265800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d7b19-887d-4d6e-a8ad-92b8319f23d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:15.266479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d7b19-887d-4d6e-a8ad-92b8319f23d4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.266772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d7b19-887d-4d6e-a8ad-92b8319f23d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.267034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d7b19-887d-4d6e-a8ad-92b8319f23d4 None admin] PATH_INFO: `/v3/projects/395264e10f8345d89e730d4c1a103dd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.269159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d7b19-887d-4d6e-a8ad-92b8319f23d4 None admin] RBAC: Authorizing `identity:get_project(project_id=395264e10f8345d89e730d4c1a103dd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:15.270648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d7b19-887d-4d6e-a8ad-92b8319f23d4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:15.272543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 221/434] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:15 2022] GET /identity/v3/projects/395264e10f8345d89e730d4c1a103dd2 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:15.273473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ded320-305c-4c2d-9982-d9d475a02e8d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.273752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ded320-305c-4c2d-9982-d9d475a02e8d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.274020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ded320-305c-4c2d-9982-d9d475a02e8d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.275553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 222/435] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:15.282925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cdcae-7f95-4414-aa1a-b6afa700f67b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.283137 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cdcae-7f95-4414-aa1a-b6afa700f67b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.283409 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cdcae-7f95-4414-aa1a-b6afa700f67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.286155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069763a-a63f-4151-8538-bd0fba74eaf1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:15.305587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9f9cdcae-7f95-4414-aa1a-b6afa700f67b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:15.306126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9f9cdcae-7f95-4414-aa1a-b6afa700f67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:15.331901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b3df0-74a4-45ae-9ed7-1e8df95c2b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:15.332656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b3df0-74a4-45ae-9ed7-1e8df95c2b51 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.332937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b3df0-74a4-45ae-9ed7-1e8df95c2b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.333210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b3df0-74a4-45ae-9ed7-1e8df95c2b51 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.334754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b3df0-74a4-45ae-9ed7-1e8df95c2b51 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:15.336097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b3df0-74a4-45ae-9ed7-1e8df95c2b51 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:15.339406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 214/436] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:15 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:15.340022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b3df0-74a4-45ae-9ed7-1e8df95c2b51 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:15.399951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:15.402340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 223/437] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:15.403379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cdcae-7f95-4414-aa1a-b6afa700f67b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:15.472172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b3df0-74a4-45ae-9ed7-1e8df95c2b51 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:15.508641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bcb48-cabb-458d-92fd-0697571c34a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:15.509338 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bcb48-cabb-458d-92fd-0697571c34a2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.509617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bcb48-cabb-458d-92fd-0697571c34a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.509890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bcb48-cabb-458d-92fd-0697571c34a2 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.511056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bcb48-cabb-458d-92fd-0697571c34a2 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:15.512483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bcb48-cabb-458d-92fd-0697571c34a2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:15.514109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 215/438] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:15 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:15.519151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bcb48-cabb-458d-92fd-0697571c34a2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:15.523351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cdcae-7f95-4414-aa1a-b6afa700f67b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:15.551054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:15.564487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d17957-676e-433b-874c-0f392c229fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:15.565384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d17957-676e-433b-874c-0f392c229fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.565883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d17957-676e-433b-874c-0f392c229fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.566226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d17957-676e-433b-874c-0f392c229fc3 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.567459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d17957-676e-433b-874c-0f392c229fc3 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:15.568877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d17957-676e-433b-874c-0f392c229fc3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:15.575176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 224/439] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:15 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:15.575176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d17957-676e-433b-874c-0f392c229fc3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:15.629609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bcb48-cabb-458d-92fd-0697571c34a2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:15.640130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:15.667532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e004f6-c78b-4b94-ac97-d12132fdb699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:15.668106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e004f6-c78b-4b94-ac97-d12132fdb699 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.668389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e004f6-c78b-4b94-ac97-d12132fdb699 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.668664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e004f6-c78b-4b94-ac97-d12132fdb699 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.681486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e004f6-c78b-4b94-ac97-d12132fdb699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-41e004f6-c78b-4b94-ac97-d12132fdb699 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:15.687391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:15.688862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:15.689877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:15.691757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d17957-676e-433b-874c-0f392c229fc3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:15.693492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 216/440] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:15.694347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e004f6-c78b-4b94-ac97-d12132fdb699 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:15.730218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1eb22-f708-46ce-9c25-8079c03cbd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:15.730963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1eb22-f708-46ce-9c25-8079c03cbd3e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.731340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1eb22-f708-46ce-9c25-8079c03cbd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.731645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1eb22-f708-46ce-9c25-8079c03cbd3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.732494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1eb22-f708-46ce-9c25-8079c03cbd3e None admin] RBAC: Adding query filter params (group.id=2aa48136483f469a9c3138ee29808008, role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=395264e10f8345d89e730d4c1a103dd2) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:15.733615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1eb22-f708-46ce-9c25-8079c03cbd3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:15.735254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1eb22-f708-46ce-9c25-8079c03cbd3e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:15.742546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 225/441] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:15 2022] GET /identity/v3/role_assignments?group.id=2aa48136483f469a9c3138ee29808008&scope.project.id=395264e10f8345d89e730d4c1a103dd2&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:15.744142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1eb22-f708-46ce-9c25-8079c03cbd3e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:15.818368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:15.818791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e004f6-c78b-4b94-ac97-d12132fdb699 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:15.858747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea30f2-ca61-4444-bece-83b56eb5ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:15.859481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea30f2-ca61-4444-bece-83b56eb5ab66 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.859832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea30f2-ca61-4444-bece-83b56eb5ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.860186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea30f2-ca61-4444-bece-83b56eb5ab66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.861067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea30f2-ca61-4444-bece-83b56eb5ab66 None admin] RBAC: Adding query filter params (group.id=2aa48136483f469a9c3138ee29808008, role.id=3d81447d2d66443384ce9ae7dd5ea35d, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:15.861454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea30f2-ca61-4444-bece-83b56eb5ab66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:15.863054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea30f2-ca61-4444-bece-83b56eb5ab66 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:15.877326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 217/442] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:15 2022] GET /identity/v3/role_assignments?group.id=2aa48136483f469a9c3138ee29808008&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=3d81447d2d66443384ce9ae7dd5ea35d => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:15.877326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eb63f-a1f1-4452-af67-db373222a764 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.877326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eb63f-a1f1-4452-af67-db373222a764 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.877326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eb63f-a1f1-4452-af67-db373222a764 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.877326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 218/443] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:15.879759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4f89-229a-4cc0-b225-a872f953bf63 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.879759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4f89-229a-4cc0-b225-a872f953bf63 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.880010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4f89-229a-4cc0-b225-a872f953bf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.883036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1eb22-f708-46ce-9c25-8079c03cbd3e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:15.901232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0a1d4f89-229a-4cc0-b225-a872f953bf63 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:15.902816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0a1d4f89-229a-4cc0-b225-a872f953bf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:15.935761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2860686-240f-4e6e-9c86-fb03b49a2b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:15.936554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2860686-240f-4e6e-9c86-fb03b49a2b90 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:15.936843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2860686-240f-4e6e-9c86-fb03b49a2b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:15.937181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2860686-240f-4e6e-9c86-fb03b49a2b90 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:15.937941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2860686-240f-4e6e-9c86-fb03b49a2b90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:15.938274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2860686-240f-4e6e-9c86-fb03b49a2b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:15.940548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:15.942195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2860686-240f-4e6e-9c86-fb03b49a2b90 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:15.952645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 226/444] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:15.962215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2860686-240f-4e6e-9c86-fb03b49a2b90 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:16.018503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 219/445] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:16.033174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d4f89-229a-4cc0-b225-a872f953bf63 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:16.107202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2860686-240f-4e6e-9c86-fb03b49a2b90 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:16.128622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:16.156967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4510a-2dac-4b2b-98d0-5703096d8f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:16.157594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4510a-2dac-4b2b-98d0-5703096d8f98 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:16.157870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4510a-2dac-4b2b-98d0-5703096d8f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:16.158142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4510a-2dac-4b2b-98d0-5703096d8f98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:16.162146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d4f89-229a-4cc0-b225-a872f953bf63 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:16.200891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219457a-c599-449e-9c37-1c17e90c5acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:16.202079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219457a-c599-449e-9c37-1c17e90c5acd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:16.202363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219457a-c599-449e-9c37-1c17e90c5acd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:16.202741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219457a-c599-449e-9c37-1c17e90c5acd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:16.216276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219457a-c599-449e-9c37-1c17e90c5acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d219457a-c599-449e-9c37-1c17e90c5acd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:16.221992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:16.223008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:16.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 220/446] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:16.229909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219457a-c599-449e-9c37-1c17e90c5acd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:16.239594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4510a-2dac-4b2b-98d0-5703096d8f98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:16.241064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4510a-2dac-4b2b-98d0-5703096d8f98 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:16.257730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:16.346734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219457a-c599-449e-9c37-1c17e90c5acd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:16.362758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 227/447] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:45:16.368375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4510a-2dac-4b2b-98d0-5703096d8f98 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:16.385235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75faf19-94c0-406c-9116-9f4e6895a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:16.385859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75faf19-94c0-406c-9116-9f4e6895a93b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:16.386144 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75faf19-94c0-406c-9116-9f4e6895a93b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:16.386282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:16.386419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75faf19-94c0-406c-9116-9f4e6895a93b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:16.387276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75faf19-94c0-406c-9116-9f4e6895a93b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:16.387606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75faf19-94c0-406c-9116-9f4e6895a93b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:16.389818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75faf19-94c0-406c-9116-9f4e6895a93b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:16.398186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 221/448] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:16.404136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75faf19-94c0-406c-9116-9f4e6895a93b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:16.470231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4510a-2dac-4b2b-98d0-5703096d8f98 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:16.491732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:16.504503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9cf1a-45f7-47ed-9d35-73a839745980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:16.505136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9cf1a-45f7-47ed-9d35-73a839745980 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:16.505412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9cf1a-45f7-47ed-9d35-73a839745980 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:16.505680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9cf1a-45f7-47ed-9d35-73a839745980 None admin] PATH_INFO: `/v3/projects/733b6ca2bdf14132a5de09c5371d80ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:16.506843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9cf1a-45f7-47ed-9d35-73a839745980 None admin] RBAC: Authorizing `identity:get_project(project_id=733b6ca2bdf14132a5de09c5371d80ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:16.507642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75faf19-94c0-406c-9116-9f4e6895a93b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:16.508297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9cf1a-45f7-47ed-9d35-73a839745980 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:16.509926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 228/449] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:16 2022] GET /identity/v3/projects/733b6ca2bdf14132a5de09c5371d80ae => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:16.515384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9cf1a-45f7-47ed-9d35-73a839745980 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:16.541627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb8fee-1e10-4133-9098-45c36f7229da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:16.542230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb8fee-1e10-4133-9098-45c36f7229da None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:16.542539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb8fee-1e10-4133-9098-45c36f7229da None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:16.542814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb8fee-1e10-4133-9098-45c36f7229da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:16.599823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:16.611188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb8fee-1e10-4133-9098-45c36f7229da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:16.612656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb8fee-1e10-4133-9098-45c36f7229da None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:16.616398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9cf1a-45f7-47ed-9d35-73a839745980 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:16.650074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f7118-968c-4f77-b336-011d67737146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:16.650735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f7118-968c-4f77-b336-011d67737146 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:16.651015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f7118-968c-4f77-b336-011d67737146 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:16.651285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f7118-968c-4f77-b336-011d67737146 None admin] PATH_INFO: `/v3/groups/2aa48136483f469a9c3138ee29808008` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:16.652691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f7118-968c-4f77-b336-011d67737146 None admin] RBAC: Authorizing `identity:get_group(group_id=2aa48136483f469a9c3138ee29808008)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:16.654110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f7118-968c-4f77-b336-011d67737146 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:16.656171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 229/450] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:16 2022] GET /identity/v3/groups/2aa48136483f469a9c3138ee29808008 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:16.661592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f7118-968c-4f77-b336-011d67737146 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:16.697173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:16.714392 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 222/451] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:45:16.720628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb8fee-1e10-4133-9098-45c36f7229da None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:16.761403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f7118-968c-4f77-b336-011d67737146 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:16.796294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182f7d3-de68-4be5-b8e7-ba47ba7b0027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:16.796900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182f7d3-de68-4be5-b8e7-ba47ba7b0027 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:16.797174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182f7d3-de68-4be5-b8e7-ba47ba7b0027 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:16.797452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182f7d3-de68-4be5-b8e7-ba47ba7b0027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:16.798228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182f7d3-de68-4be5-b8e7-ba47ba7b0027 None admin] RBAC: Adding query filter params (group.id=2aa48136483f469a9c3138ee29808008, role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=733b6ca2bdf14132a5de09c5371d80ae) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:16.798565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182f7d3-de68-4be5-b8e7-ba47ba7b0027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:16.799989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182f7d3-de68-4be5-b8e7-ba47ba7b0027 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:16.805604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 230/452] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:16 2022] GET /identity/v3/role_assignments?group.id=2aa48136483f469a9c3138ee29808008&scope.project.id=733b6ca2bdf14132a5de09c5371d80ae&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:16.818855 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb8fee-1e10-4133-9098-45c36f7229da None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:16.857541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931811d0-896d-4734-84e1-5e60440ed9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:16.858244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931811d0-896d-4734-84e1-5e60440ed9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:16.858590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931811d0-896d-4734-84e1-5e60440ed9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:16.858837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931811d0-896d-4734-84e1-5e60440ed9c8 None admin] PATH_INFO: `/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:16.860106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931811d0-896d-4734-84e1-5e60440ed9c8 None admin] RBAC: Authorizing `identity:get_project(project_id=2e75ff1f24d24ead87dfc7d32ddc1723)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:16.861582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931811d0-896d-4734-84e1-5e60440ed9c8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:16.863556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 223/453] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:45:16 2022] GET /identity/v3/projects/2e75ff1f24d24ead87dfc7d32ddc1723 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:16.869176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182f7d3-de68-4be5-b8e7-ba47ba7b0027 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:16.889857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:16.978055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182f7d3-de68-4be5-b8e7-ba47ba7b0027 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:17.014890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01168dc2-4d5d-4d17-8ed6-7de56769f7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:17.015814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01168dc2-4d5d-4d17-8ed6-7de56769f7dd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:17.016090 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01168dc2-4d5d-4d17-8ed6-7de56769f7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:17.016360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01168dc2-4d5d-4d17-8ed6-7de56769f7dd None admin] PATH_INFO: `/v3/groups/f2106ba5551149e4af3daf5c0406e263` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:17.017509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01168dc2-4d5d-4d17-8ed6-7de56769f7dd None admin] RBAC: Authorizing `identity:get_group(group_id=f2106ba5551149e4af3daf5c0406e263)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:17.018982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01168dc2-4d5d-4d17-8ed6-7de56769f7dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:17.020911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 231/454] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Jul 30 04:45:16 2022] GET /identity/v3/groups/f2106ba5551149e4af3daf5c0406e263 => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:17.026808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931811d0-896d-4734-84e1-5e60440ed9c8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:17.084494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:17.134520 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931811d0-896d-4734-84e1-5e60440ed9c8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:17.169034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761d388-5df4-46b0-bf13-ae73fbde580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:17.169640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761d388-5df4-46b0-bf13-ae73fbde580c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:17.169913 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761d388-5df4-46b0-bf13-ae73fbde580c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:17.170222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761d388-5df4-46b0-bf13-ae73fbde580c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:17.171193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761d388-5df4-46b0-bf13-ae73fbde580c None admin] RBAC: Adding query filter params (group.id=f2106ba5551149e4af3daf5c0406e263, role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=2e75ff1f24d24ead87dfc7d32ddc1723) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:17.171499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761d388-5df4-46b0-bf13-ae73fbde580c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:17.172912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761d388-5df4-46b0-bf13-ae73fbde580c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:17.178650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 224/455] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Jul 30 04:45:17 2022] GET /identity/v3/role_assignments?group.id=f2106ba5551149e4af3daf5c0406e263&scope.project.id=2e75ff1f24d24ead87dfc7d32ddc1723&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:19.581403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c78398-91b4-45eb-95fd-53edd47e519e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:19.581916 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c78398-91b4-45eb-95fd-53edd47e519e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:19.582030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c78398-91b4-45eb-95fd-53edd47e519e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:19.583608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 232/456] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:19.589250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364f048-07ba-4af4-b2ff-359b29987142 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:19.589994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364f048-07ba-4af4-b2ff-359b29987142 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:19.590486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364f048-07ba-4af4-b2ff-359b29987142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:19.614862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0364f048-07ba-4af4-b2ff-359b29987142 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:19.616414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0364f048-07ba-4af4-b2ff-359b29987142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:19.701243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86727ef7-ea20-472a-8d18-2daccb7f44f4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:19.702992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86727ef7-ea20-472a-8d18-2daccb7f44f4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:19.703475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86727ef7-ea20-472a-8d18-2daccb7f44f4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:19.704505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 225/457] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:19.705519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 233/458] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:19.712162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb595e34-cec1-4ae0-8d99-8026a5b4d74b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:19.712529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb595e34-cec1-4ae0-8d99-8026a5b4d74b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:19.712811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb595e34-cec1-4ae0-8d99-8026a5b4d74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:19.731753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86727ef7-ea20-472a-8d18-2daccb7f44f4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:19.734153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bb595e34-cec1-4ae0-8d99-8026a5b4d74b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:19.735457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bb595e34-cec1-4ae0-8d99-8026a5b4d74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:19.791636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:19.837688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 226/459] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:19.865253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb595e34-cec1-4ae0-8d99-8026a5b4d74b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:19.895985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:19.936526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86727ef7-ea20-472a-8d18-2daccb7f44f4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:20.008938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a6b37-23b9-4c7d-862d-9bfad7d332e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:20.009595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a6b37-23b9-4c7d-862d-9bfad7d332e0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.009878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a6b37-23b9-4c7d-862d-9bfad7d332e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.010155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a6b37-23b9-4c7d-862d-9bfad7d332e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.017206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb595e34-cec1-4ae0-8d99-8026a5b4d74b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:20.036970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a6b37-23b9-4c7d-862d-9bfad7d332e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:20.038477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a6b37-23b9-4c7d-862d-9bfad7d332e0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b33a6b37-23b9-4c7d-862d-9bfad7d332e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:20.056233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:20.057149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:20.058096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:20.058096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:20.058096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:20.058096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:20.058096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 234/460] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:20.059100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f5537-60e2-4b6b-8363-cd9c73063a48 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.059386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f5537-60e2-4b6b-8363-cd9c73063a48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.059664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f5537-60e2-4b6b-8363-cd9c73063a48 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.061586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 235/461] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:20.064463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f5537-60e2-4b6b-8363-cd9c73063a48 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:20.074276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314cbe7-0abf-4545-adad-112b8195790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:20.075010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314cbe7-0abf-4545-adad-112b8195790c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.075292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314cbe7-0abf-4545-adad-112b8195790c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.075565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314cbe7-0abf-4545-adad-112b8195790c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.087258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314cbe7-0abf-4545-adad-112b8195790c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:20.088800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314cbe7-0abf-4545-adad-112b8195790c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d314cbe7-0abf-4545-adad-112b8195790c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:20.105440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:20.106341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:20.107249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:20.107249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:20.107249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:20.107249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:20.108007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 227/462] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:20.109601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b25cf3-9018-4aae-bf99-996320fc5612 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.109950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b25cf3-9018-4aae-bf99-996320fc5612 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.110298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b25cf3-9018-4aae-bf99-996320fc5612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.129151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c6b25cf3-9018-4aae-bf99-996320fc5612 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:20.130470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c6b25cf3-9018-4aae-bf99-996320fc5612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:20.156591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:20.190777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f5537-60e2-4b6b-8363-cd9c73063a48 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:20.210616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 228/463] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:20.211748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b25cf3-9018-4aae-bf99-996320fc5612 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:20.231789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301b53d-4a65-4af0-9457-432d5a02b7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:20.232743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301b53d-4a65-4af0-9457-432d5a02b7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.233606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301b53d-4a65-4af0-9457-432d5a02b7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.234110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301b53d-4a65-4af0-9457-432d5a02b7c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.235035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301b53d-4a65-4af0-9457-432d5a02b7c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:20.235542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301b53d-4a65-4af0-9457-432d5a02b7c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:20.237143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301b53d-4a65-4af0-9457-432d5a02b7c8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:20.240083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:20.253014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 236/464] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:20.258731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301b53d-4a65-4af0-9457-432d5a02b7c8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:20.340598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b25cf3-9018-4aae-bf99-996320fc5612 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:20.374810 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301b53d-4a65-4af0-9457-432d5a02b7c8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:20.376768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d28301-f414-448c-bf21-90ebf16e3151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:20.377393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d28301-f414-448c-bf21-90ebf16e3151 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.377677 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d28301-f414-448c-bf21-90ebf16e3151 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.377953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d28301-f414-448c-bf21-90ebf16e3151 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.379420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d28301-f414-448c-bf21-90ebf16e3151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:20.379420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d28301-f414-448c-bf21-90ebf16e3151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:20.380362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d28301-f414-448c-bf21-90ebf16e3151 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:20.393423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 229/465] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:20.394324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d28301-f414-448c-bf21-90ebf16e3151 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:20.414474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:20.416279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8008782-eef4-4939-83fa-f7bb4bed9ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:20.416911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8008782-eef4-4939-83fa-f7bb4bed9ab4 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.417190 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8008782-eef4-4939-83fa-f7bb4bed9ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.417469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8008782-eef4-4939-83fa-f7bb4bed9ab4 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.418134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8008782-eef4-4939-83fa-f7bb4bed9ab4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:20.419622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8008782-eef4-4939-83fa-f7bb4bed9ab4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:20.445795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 237/466] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:45:20 2022] POST /identity/v3/users => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:20.447318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8008782-eef4-4939-83fa-f7bb4bed9ab4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:20.520301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d28301-f414-448c-bf21-90ebf16e3151 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:20.537264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:20.566252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c40cf-2a78-4728-a898-a6e32af36d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:20.567091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c40cf-2a78-4728-a898-a6e32af36d2b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.567447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c40cf-2a78-4728-a898-a6e32af36d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.567800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c40cf-2a78-4728-a898-a6e32af36d2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.582459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c40cf-2a78-4728-a898-a6e32af36d2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:20.584062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c40cf-2a78-4728-a898-a6e32af36d2b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-af4c40cf-2a78-4728-a898-a6e32af36d2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:20.602818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:20.604118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:20.604956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:20.604956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:20.604956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:20.604956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:20.606504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8008782-eef4-4939-83fa-f7bb4bed9ab4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:20.609590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 230/467] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:20.611393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcc308-2de7-4276-b5f5-af014d061ef9 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.611686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcc308-2de7-4276-b5f5-af014d061ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.611963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcc308-2de7-4276-b5f5-af014d061ef9 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.613365 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 231/468] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:20.614099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcc308-2de7-4276-b5f5-af014d061ef9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:20.652613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f279b5-d053-4bda-be05-9d3cc220cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:20.653339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f279b5-d053-4bda-be05-9d3cc220cac4 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.653614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f279b5-d053-4bda-be05-9d3cc220cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.653883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f279b5-d053-4bda-be05-9d3cc220cac4 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.656021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f279b5-d053-4bda-be05-9d3cc220cac4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:20.656991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f279b5-d053-4bda-be05-9d3cc220cac4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:20.689537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 238/469] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:45:20 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:20.691326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b8f78-98be-46f8-8ebd-4dfe52355ba8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.691878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b8f78-98be-46f8-8ebd-4dfe52355ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.692410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b8f78-98be-46f8-8ebd-4dfe52355ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.708201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:20.715483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8c4b8f78-98be-46f8-8ebd-4dfe52355ba8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:20.717772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c4b8f78-98be-46f8-8ebd-4dfe52355ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:20.749926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcc308-2de7-4276-b5f5-af014d061ef9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:20.794794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4bac7-5558-4a3f-b2a3-f7b80af6e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:20.796204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4bac7-5558-4a3f-b2a3-f7b80af6e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:20.796667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4bac7-5558-4a3f-b2a3-f7b80af6e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:20.797120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4bac7-5558-4a3f-b2a3-f7b80af6e3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:20.798050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4bac7-5558-4a3f-b2a3-f7b80af6e3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:20.798371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4bac7-5558-4a3f-b2a3-f7b80af6e3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:20.799832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4bac7-5558-4a3f-b2a3-f7b80af6e3f5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:20.818016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 232/470] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:20.824544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4bac7-5558-4a3f-b2a3-f7b80af6e3f5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:20.837496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 239/471] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:20.873089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b8f78-98be-46f8-8ebd-4dfe52355ba8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:20.938196 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:20.991950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4bac7-5558-4a3f-b2a3-f7b80af6e3f5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:21.022792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b8f78-98be-46f8-8ebd-4dfe52355ba8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:21.023261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:21.032487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbfb18-f8e5-4395-bf8c-8e0513c1b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:21.033196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbfb18-f8e5-4395-bf8c-8e0513c1b606 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:21.033539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbfb18-f8e5-4395-bf8c-8e0513c1b606 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:21.033884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbfb18-f8e5-4395-bf8c-8e0513c1b606 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:21.034785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbfb18-f8e5-4395-bf8c-8e0513c1b606 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:21.036306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbfb18-f8e5-4395-bf8c-8e0513c1b606 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:21.063652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 233/472] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:45:20 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:21.064314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c4498-74b8-4c5c-803e-a08e9c50b432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:21.064968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c4498-74b8-4c5c-803e-a08e9c50b432 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:21.065247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c4498-74b8-4c5c-803e-a08e9c50b432 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:21.065515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c4498-74b8-4c5c-803e-a08e9c50b432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:21.080948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c4498-74b8-4c5c-803e-a08e9c50b432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:21.082480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c4498-74b8-4c5c-803e-a08e9c50b432 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-424c4498-74b8-4c5c-803e-a08e9c50b432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:21.101410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:21.102385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:21.103352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:21.103352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:21.103352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:21.103352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:21.103352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 240/473] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:21.110740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbfb18-f8e5-4395-bf8c-8e0513c1b606 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:21.191408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29297d7c-2105-4652-927e-7127ac936344 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:21.191587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29297d7c-2105-4652-927e-7127ac936344 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:21.191855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29297d7c-2105-4652-927e-7127ac936344 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:21.193293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 241/474] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:45:21.199164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab67326-8cf3-4338-92f3-852b255c3cb3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:21.199444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab67326-8cf3-4338-92f3-852b255c3cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:21.199720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab67326-8cf3-4338-92f3-852b255c3cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:21.226534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eab67326-8cf3-4338-92f3-852b255c3cb3 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:21.227776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eab67326-8cf3-4338-92f3-852b255c3cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:21.262181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbfb18-f8e5-4395-bf8c-8e0513c1b606 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:21.304865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98400c-e91c-4ee4-986d-b1eab7679b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:21.305522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98400c-e91c-4ee4-986d-b1eab7679b3f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:21.305802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98400c-e91c-4ee4-986d-b1eab7679b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:21.306075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98400c-e91c-4ee4-986d-b1eab7679b3f None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:21.306917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98400c-e91c-4ee4-986d-b1eab7679b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:21.307224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98400c-e91c-4ee4-986d-b1eab7679b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:21.308624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98400c-e91c-4ee4-986d-b1eab7679b3f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:21.326810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 234/475] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:21.330640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98400c-e91c-4ee4-986d-b1eab7679b3f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:21.332138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 242/476] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:21.355255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab67326-8cf3-4338-92f3-852b255c3cb3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:21.383781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:21.385660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:21.443320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98400c-e91c-4ee4-986d-b1eab7679b3f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:21.475918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab67326-8cf3-4338-92f3-852b255c3cb3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:21.482249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368c09-3114-4b31-8e5a-0919434faf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:21.483024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368c09-3114-4b31-8e5a-0919434faf05 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:21.483299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368c09-3114-4b31-8e5a-0919434faf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:21.483568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368c09-3114-4b31-8e5a-0919434faf05 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:21.484317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368c09-3114-4b31-8e5a-0919434faf05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:21.485770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368c09-3114-4b31-8e5a-0919434faf05 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:21.511839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 235/477] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:45:21 2022] POST /identity/v3/users => generated 249 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:21.527675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:21.528335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:21.528615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:21.528884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:21.530313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:21.531893 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:21.552283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 30 04:45:21.552283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 30 04:45:21.552283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [parameters: {'id': '3081ee84e3894e44a904b62b799782b2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 30 04:45:21.552283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 30 04:45:21.570252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return ref.to_dict() Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 30 04:45:21.571265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 30 04:45:21.572463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 30 04:45:21.573307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 30 04:45:21.574160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 30 04:45:21.575061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': '3081ee84e3894e44a904b62b799782b2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:21.575851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 30 04:45:21.577050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:21.578030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 243/478] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Jul 30 04:45:21 2022] POST /identity/v3/roles => generated 144 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:21.585952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368c09-3114-4b31-8e5a-0919434faf05 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:21.735954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368c09-3114-4b31-8e5a-0919434faf05 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:21.737502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:21.772452 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07a440-0802-4a3d-983b-af3c3632c54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:21.773102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07a440-0802-4a3d-983b-af3c3632c54f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:21.773379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07a440-0802-4a3d-983b-af3c3632c54f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:21.773648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07a440-0802-4a3d-983b-af3c3632c54f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:21.784469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07a440-0802-4a3d-983b-af3c3632c54f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3e07a440-0802-4a3d-983b-af3c3632c54f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:21.789918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:21.790860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:21.791873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 236/479] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:21.798898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:21.890581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:21.907548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bdcde-3eba-490e-a0db-677497da50b7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:21.943885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f401d-9ad9-43c8-a47c-df47aa1e75c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:21.944568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f401d-9ad9-43c8-a47c-df47aa1e75c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:21.944846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f401d-9ad9-43c8-a47c-df47aa1e75c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:21.945117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f401d-9ad9-43c8-a47c-df47aa1e75c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:21.945888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f401d-9ad9-43c8-a47c-df47aa1e75c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:21.946190 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f401d-9ad9-43c8-a47c-df47aa1e75c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:21.947587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f401d-9ad9-43c8-a47c-df47aa1e75c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:21.956650 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 244/480] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:23.053814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9574-cb34-47be-ba12-a000fb31432d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:23.054474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9574-cb34-47be-ba12-a000fb31432d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:23.054474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9574-cb34-47be-ba12-a000fb31432d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:23.056072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 237/481] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:23.061807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaf2c9-9d51-4785-a5f1-f50e39d8bb4f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:23.062324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaf2c9-9d51-4785-a5f1-f50e39d8bb4f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:23.062538 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaf2c9-9d51-4785-a5f1-f50e39d8bb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:23.081100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ccdaf2c9-9d51-4785-a5f1-f50e39d8bb4f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:23.082316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ccdaf2c9-9d51-4785-a5f1-f50e39d8bb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:23.154105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 245/482] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:23.177659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9574-cb34-47be-ba12-a000fb31432d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:23.309037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9574-cb34-47be-ba12-a000fb31432d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:23.343962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35e23b-e23f-4a0a-9d56-7bea2a9f5dd6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:23.344275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35e23b-e23f-4a0a-9d56-7bea2a9f5dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:23.344511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35e23b-e23f-4a0a-9d56-7bea2a9f5dd6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:23.346171 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f546ca-3a94-49de-93ba-6cae0786a109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:23.346459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 246/483] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:23.346850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f546ca-3a94-49de-93ba-6cae0786a109 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:23.347126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f546ca-3a94-49de-93ba-6cae0786a109 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:23.347393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f546ca-3a94-49de-93ba-6cae0786a109 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:23.355497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef5329-b6d8-4e54-ac1d-24ee6126fe06 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:23.355884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef5329-b6d8-4e54-ac1d-24ee6126fe06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:23.356166 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef5329-b6d8-4e54-ac1d-24ee6126fe06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:23.361974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f546ca-3a94-49de-93ba-6cae0786a109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e4f546ca-3a94-49de-93ba-6cae0786a109 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:23.366934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:23.369044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:23.370101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 238/484] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:23.373962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f546ca-3a94-49de-93ba-6cae0786a109 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:23.395411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5eef5329-b6d8-4e54-ac1d-24ee6126fe06 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:23.396608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5eef5329-b6d8-4e54-ac1d-24ee6126fe06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:23.405831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:23.487366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f546ca-3a94-49de-93ba-6cae0786a109 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:23.526179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f487d2-7c53-47b6-babc-640a25128d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:23.527089 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f487d2-7c53-47b6-babc-640a25128d47 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:23.527369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f487d2-7c53-47b6-babc-640a25128d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:23.527645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f487d2-7c53-47b6-babc-640a25128d47 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:23.528367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f487d2-7c53-47b6-babc-640a25128d47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:23.528668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f487d2-7c53-47b6-babc-640a25128d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:23.530779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f487d2-7c53-47b6-babc-640a25128d47 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:23.533163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 247/485] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:23.549766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 239/486] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:23.554957 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef5329-b6d8-4e54-ac1d-24ee6126fe06 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:23.574863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:23.578191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f487d2-7c53-47b6-babc-640a25128d47 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:23.678532 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef5329-b6d8-4e54-ac1d-24ee6126fe06 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:23.716812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2cd98-5ff6-4828-979d-9c40e4b45f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:23.718033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2cd98-5ff6-4828-979d-9c40e4b45f0c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:23.718308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2cd98-5ff6-4828-979d-9c40e4b45f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:23.718618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2cd98-5ff6-4828-979d-9c40e4b45f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:23.729471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2cd98-5ff6-4828-979d-9c40e4b45f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:23.731106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2cd98-5ff6-4828-979d-9c40e4b45f0c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:23.746348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-35e2cd98-5ff6-4828-979d-9c40e4b45f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:23.749324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:23.750364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:23.751282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:23.751282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:23.751282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:23.751282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:23.751282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 248/487] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:23.757022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2cd98-5ff6-4828-979d-9c40e4b45f0c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:23.783915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f487d2-7c53-47b6-babc-640a25128d47 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:23.823551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:23.869153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2cd98-5ff6-4828-979d-9c40e4b45f0c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:23.876572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd884ba9-dcc5-4098-afe4-a1d098770eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:23.877239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd884ba9-dcc5-4098-afe4-a1d098770eda None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:23.877519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd884ba9-dcc5-4098-afe4-a1d098770eda None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:23.877792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd884ba9-dcc5-4098-afe4-a1d098770eda None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:23.907758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5eebc-23b6-4470-a517-6e85d10562d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:23.908404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5eebc-23b6-4470-a517-6e85d10562d9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:23.908692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5eebc-23b6-4470-a517-6e85d10562d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:23.908973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5eebc-23b6-4470-a517-6e85d10562d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:23.909674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5eebc-23b6-4470-a517-6e85d10562d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:23.909984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5eebc-23b6-4470-a517-6e85d10562d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:23.911556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5eebc-23b6-4470-a517-6e85d10562d9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:23.915038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd884ba9-dcc5-4098-afe4-a1d098770eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-bd884ba9-dcc5-4098-afe4-a1d098770eda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:23.920578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:23.921488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:23.922312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:23.924203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 249/489] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:23.924314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 240/489] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:23.929244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5eebc-23b6-4470-a517-6e85d10562d9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:23.929371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd884ba9-dcc5-4098-afe4-a1d098770eda None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:24.040575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd884ba9-dcc5-4098-afe4-a1d098770eda None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:24.065577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5eebc-23b6-4470-a517-6e85d10562d9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:24.085781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:24.087476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10e52b-4999-45c3-ba56-d788fedcdbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:24.088136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10e52b-4999-45c3-ba56-d788fedcdbdb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.088431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10e52b-4999-45c3-ba56-d788fedcdbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.088694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10e52b-4999-45c3-ba56-d788fedcdbdb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.100465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10e52b-4999-45c3-ba56-d788fedcdbdb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:24.102615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10e52b-4999-45c3-ba56-d788fedcdbdb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:24.107074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e36328-57df-4466-af50-90ffc015824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:24.107912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e36328-57df-4466-af50-90ffc015824a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.108192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e36328-57df-4466-af50-90ffc015824a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.108471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e36328-57df-4466-af50-90ffc015824a None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.109234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e36328-57df-4466-af50-90ffc015824a None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:24.109542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e36328-57df-4466-af50-90ffc015824a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:24.110943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e36328-57df-4466-af50-90ffc015824a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ea10e52b-4999-45c3-ba56-d788fedcdbdb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:24.119493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:24.120385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:24.121290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:24.128682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 250/491] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:24.128682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018cd81-4ba3-44c8-92cd-1993e648e822 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.128682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018cd81-4ba3-44c8-92cd-1993e648e822 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.128682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018cd81-4ba3-44c8-92cd-1993e648e822 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.129017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 251/492] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:24.129269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 241/492] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Jul 30 04:45:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:24.129787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018cd81-4ba3-44c8-92cd-1993e648e822 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:24.130721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10e52b-4999-45c3-ba56-d788fedcdbdb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:24.136167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:24.244303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018cd81-4ba3-44c8-92cd-1993e648e822 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:24.256666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10e52b-4999-45c3-ba56-d788fedcdbdb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:24.282301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e336a-1ebe-468c-bb78-a79e53894529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:24.283319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e336a-1ebe-468c-bb78-a79e53894529 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.283914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e336a-1ebe-468c-bb78-a79e53894529 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.284397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e336a-1ebe-468c-bb78-a79e53894529 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.285501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e336a-1ebe-468c-bb78-a79e53894529 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:24.285959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e336a-1ebe-468c-bb78-a79e53894529 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:24.287851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e336a-1ebe-468c-bb78-a79e53894529 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:24.292098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a62369-47ae-47a1-9d91-d02b8c7d7206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:24.292731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a62369-47ae-47a1-9d91-d02b8c7d7206 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.293007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a62369-47ae-47a1-9d91-d02b8c7d7206 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.293276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a62369-47ae-47a1-9d91-d02b8c7d7206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.305152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a62369-47ae-47a1-9d91-d02b8c7d7206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:24.306647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a62369-47ae-47a1-9d91-d02b8c7d7206 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c3a62369-47ae-47a1-9d91-d02b8c7d7206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:24.321542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:24.322632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:24.323519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:24.323519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:24.323519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:24.323519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:24.323519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 242/493] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:24.323854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019210da-26f9-42f8-adf6-00ad27c1c475 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.324135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019210da-26f9-42f8-adf6-00ad27c1c475 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.324404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019210da-26f9-42f8-adf6-00ad27c1c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.335240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 252/494] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Jul 30 04:45:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:24.340907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de952b28-471c-41ac-9967-36dad0c72320 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.341204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de952b28-471c-41ac-9967-36dad0c72320 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.341481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de952b28-471c-41ac-9967-36dad0c72320 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.343626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-019210da-26f9-42f8-adf6-00ad27c1c475 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:24.344033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 253/495] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:24.344758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de952b28-471c-41ac-9967-36dad0c72320 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:24.348563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-019210da-26f9-42f8-adf6-00ad27c1c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:24.389993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:24.403905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:24.434460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 243/496] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:24.434460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019210da-26f9-42f8-adf6-00ad27c1c475 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:24.471089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de952b28-471c-41ac-9967-36dad0c72320 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:24.508704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f9709-068c-45e5-91e2-7ee2cdfc18bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:24.509333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f9709-068c-45e5-91e2-7ee2cdfc18bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.509615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f9709-068c-45e5-91e2-7ee2cdfc18bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.509889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f9709-068c-45e5-91e2-7ee2cdfc18bb None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.510609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f9709-068c-45e5-91e2-7ee2cdfc18bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:24.510934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f9709-068c-45e5-91e2-7ee2cdfc18bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:24.512302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f9709-068c-45e5-91e2-7ee2cdfc18bb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:24.528607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 254/497] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:24.528607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b24a4-98cf-41b6-bd12-9e8ca0e1e0ed None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.528607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b24a4-98cf-41b6-bd12-9e8ca0e1e0ed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.528607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b24a4-98cf-41b6-bd12-9e8ca0e1e0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.542960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019210da-26f9-42f8-adf6-00ad27c1c475 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:24.544582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7e7b24a4-98cf-41b6-bd12-9e8ca0e1e0ed None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:24.546588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e7b24a4-98cf-41b6-bd12-9e8ca0e1e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:24.578905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833771f4-5568-45e0-8b55-8da6dbe316cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:24.579524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833771f4-5568-45e0-8b55-8da6dbe316cf None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.579802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833771f4-5568-45e0-8b55-8da6dbe316cf None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.580071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833771f4-5568-45e0-8b55-8da6dbe316cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.622611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 255/498] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:24.622611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b24a4-98cf-41b6-bd12-9e8ca0e1e0ed None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:24.640934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:24.652804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:24.655796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833771f4-5568-45e0-8b55-8da6dbe316cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:24.657300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833771f4-5568-45e0-8b55-8da6dbe316cf None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:24.730338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b24a4-98cf-41b6-bd12-9e8ca0e1e0ed None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:24.760668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 244/499] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:24.761512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833771f4-5568-45e0-8b55-8da6dbe316cf None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:24.765170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe11c8c-dbc5-4dd4-b2de-a505044afa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:24.765821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe11c8c-dbc5-4dd4-b2de-a505044afa63 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.766098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe11c8c-dbc5-4dd4-b2de-a505044afa63 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.766369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe11c8c-dbc5-4dd4-b2de-a505044afa63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.777026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe11c8c-dbc5-4dd4-b2de-a505044afa63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8fe11c8c-dbc5-4dd4-b2de-a505044afa63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:24.782462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:24.783552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:24.784545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 256/500] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:24.786127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe11c8c-dbc5-4dd4-b2de-a505044afa63 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:24.847522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:24.863306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833771f4-5568-45e0-8b55-8da6dbe316cf None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:24.874502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:24.886306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe11c8c-dbc5-4dd4-b2de-a505044afa63 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:24.898054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eabb18-d519-4059-8d71-021b5fc9c2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:24.898659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eabb18-d519-4059-8d71-021b5fc9c2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.898942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eabb18-d519-4059-8d71-021b5fc9c2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.899218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eabb18-d519-4059-8d71-021b5fc9c2d1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.906682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eabb18-d519-4059-8d71-021b5fc9c2d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:24.908121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eabb18-d519-4059-8d71-021b5fc9c2d1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-82eabb18-d519-4059-8d71-021b5fc9c2d1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:24.919489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:24.921148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 30 04:45:24.922051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:24.922673 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e588cf-b531-49c4-9e0c-b521eab0d936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:24.922758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 245/501] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:24.922837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e588cf-b531-49c4-9e0c-b521eab0d936 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.922837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e588cf-b531-49c4-9e0c-b521eab0d936 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.922837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e588cf-b531-49c4-9e0c-b521eab0d936 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.923040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4ca9f-8368-41d2-a463-5d73b4f6d2ec None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:24.923040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4ca9f-8368-41d2-a463-5d73b4f6d2ec None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:24.923040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4ca9f-8368-41d2-a463-5d73b4f6d2ec None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:24.924241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 246/502] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:24.924932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4ca9f-8368-41d2-a463-5d73b4f6d2ec None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:24.937928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e588cf-b531-49c4-9e0c-b521eab0d936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-03e588cf-b531-49c4-9e0c-b521eab0d936 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:24.942887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:24.943963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 257/503] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:24.945094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e588cf-b531-49c4-9e0c-b521eab0d936 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.031516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4ca9f-8368-41d2-a463-5d73b4f6d2ec None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.049863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e588cf-b531-49c4-9e0c-b521eab0d936 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.064292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141979c-5b8b-4a11-93e1-acdae73a3478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.064921 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141979c-5b8b-4a11-93e1-acdae73a3478 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.065195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141979c-5b8b-4a11-93e1-acdae73a3478 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.065463 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141979c-5b8b-4a11-93e1-acdae73a3478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.076776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141979c-5b8b-4a11-93e1-acdae73a3478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.078650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141979c-5b8b-4a11-93e1-acdae73a3478 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.083650 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099f633-f948-402f-9554-c8730501ebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.084261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099f633-f948-402f-9554-c8730501ebab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.084541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099f633-f948-402f-9554-c8730501ebab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.084812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099f633-f948-402f-9554-c8730501ebab None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.085389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:25.085518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099f633-f948-402f-9554-c8730501ebab None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:25.085820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099f633-f948-402f-9554-c8730501ebab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.087221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099f633-f948-402f-9554-c8730501ebab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8141979c-5b8b-4a11-93e1-acdae73a3478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:25.094138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:25.095441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:25.096314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:25.096314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:25.096314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:25.096314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:25.096314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 247/504] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:25.096892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141979c-5b8b-4a11-93e1-acdae73a3478 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.102461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 258/505] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:25.102461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ce23d-e564-4497-8cc4-1c7bd1275980 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.102461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ce23d-e564-4497-8cc4-1c7bd1275980 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.102461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ce23d-e564-4497-8cc4-1c7bd1275980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.118999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fc7ce23d-e564-4497-8cc4-1c7bd1275980 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:25.120216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc7ce23d-e564-4497-8cc4-1c7bd1275980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:25.123105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:25.188518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 259/506] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:25.189285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ce23d-e564-4497-8cc4-1c7bd1275980 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.215894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141979c-5b8b-4a11-93e1-acdae73a3478 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.250197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cec91-0e54-4df0-87b1-907b90d67d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.250849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cec91-0e54-4df0-87b1-907b90d67d54 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.251123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cec91-0e54-4df0-87b1-907b90d67d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.251401 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cec91-0e54-4df0-87b1-907b90d67d54 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.252185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cec91-0e54-4df0-87b1-907b90d67d54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:25.252487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cec91-0e54-4df0-87b1-907b90d67d54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.253973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cec91-0e54-4df0-87b1-907b90d67d54 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.289940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ce23d-e564-4497-8cc4-1c7bd1275980 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.290664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 248/507] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:25.291497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cec91-0e54-4df0-87b1-907b90d67d54 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.325938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b9450-d938-4349-bf8b-7d48bfec32f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.327175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b9450-d938-4349-bf8b-7d48bfec32f2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.327461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b9450-d938-4349-bf8b-7d48bfec32f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.327738 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b9450-d938-4349-bf8b-7d48bfec32f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.328453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b9450-d938-4349-bf8b-7d48bfec32f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:25.328756 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b9450-d938-4349-bf8b-7d48bfec32f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.330150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b9450-d938-4349-bf8b-7d48bfec32f2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.338086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 260/508] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:25.338898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b9450-d938-4349-bf8b-7d48bfec32f2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.341402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:25.348809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:25.392142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cec91-0e54-4df0-87b1-907b90d67d54 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.426767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0627b-9609-4f21-b508-7d9eae591cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.427408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0627b-9609-4f21-b508-7d9eae591cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.427682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0627b-9609-4f21-b508-7d9eae591cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.427950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0627b-9609-4f21-b508-7d9eae591cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.428631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0627b-9609-4f21-b508-7d9eae591cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:25.428930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0627b-9609-4f21-b508-7d9eae591cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.430306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0627b-9609-4f21-b508-7d9eae591cf5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.441346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 249/509] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:25.442060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0627b-9609-4f21-b508-7d9eae591cf5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.442157 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b9450-d938-4349-bf8b-7d48bfec32f2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.478202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8fc3-71f4-4cd1-9a22-d6551cf60317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.478865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8fc3-71f4-4cd1-9a22-d6551cf60317 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.479139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8fc3-71f4-4cd1-9a22-d6551cf60317 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.479407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8fc3-71f4-4cd1-9a22-d6551cf60317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.490238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca8fc3-71f4-4cd1-9a22-d6551cf60317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.491716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca8fc3-71f4-4cd1-9a22-d6551cf60317 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-08ca8fc3-71f4-4cd1-9a22-d6551cf60317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:25.505593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:25.506495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:25.507362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:25.507362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:25.507362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:25.507362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:25.507362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 261/510] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:25.507658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8fc3-71f4-4cd1-9a22-d6551cf60317 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.546346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0627b-9609-4f21-b508-7d9eae591cf5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.562344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:25.566147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:25.581310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b1312-604d-4394-955d-88a7f0cbe0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.581951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b1312-604d-4394-955d-88a7f0cbe0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.582233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b1312-604d-4394-955d-88a7f0cbe0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.582535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b1312-604d-4394-955d-88a7f0cbe0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.593385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b1312-604d-4394-955d-88a7f0cbe0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.594866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b1312-604d-4394-955d-88a7f0cbe0a7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ba6b1312-604d-4394-955d-88a7f0cbe0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:25.609035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:25.610081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:25.611181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:25.611181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:25.611181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:25.611181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:25.611181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 250/511] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:25.611503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b1312-604d-4394-955d-88a7f0cbe0a7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.615539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8fc3-71f4-4cd1-9a22-d6551cf60317 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.650066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ed38f-5bfb-4ce9-ab3b-951fe4e3ad52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.650712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ed38f-5bfb-4ce9-ab3b-951fe4e3ad52 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.650996 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ed38f-5bfb-4ce9-ab3b-951fe4e3ad52 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.651272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ed38f-5bfb-4ce9-ab3b-951fe4e3ad52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.713368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b1312-604d-4394-955d-88a7f0cbe0a7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.720247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ed38f-5bfb-4ce9-ab3b-951fe4e3ad52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.721716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ed38f-5bfb-4ce9-ab3b-951fe4e3ad52 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.746196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f8631-8e49-4795-be13-1ca6c6d6d9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.746932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f8631-8e49-4795-be13-1ca6c6d6d9ea None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.747279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f8631-8e49-4795-be13-1ca6c6d6d9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.747619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f8631-8e49-4795-be13-1ca6c6d6d9ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.758530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f8631-8e49-4795-be13-1ca6c6d6d9ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.759977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f8631-8e49-4795-be13-1ca6c6d6d9ea None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1f3f8631-8e49-4795-be13-1ca6c6d6d9ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:25.773991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:25.775048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:25.775924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:25.775924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:25.775924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:25.775924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:25.775924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 251/512] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:25.775924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f8631-8e49-4795-be13-1ca6c6d6d9ea None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.777145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:25.801244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:25.827484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 262/513] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:25.828327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ed38f-5bfb-4ce9-ab3b-951fe4e3ad52 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.880395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f8631-8e49-4795-be13-1ca6c6d6d9ea None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.913652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7c43-fa1e-4240-9301-1f090421fff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.914288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7c43-fa1e-4240-9301-1f090421fff8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.914602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7c43-fa1e-4240-9301-1f090421fff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.914879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7c43-fa1e-4240-9301-1f090421fff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.922234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f7c43-fa1e-4240-9301-1f090421fff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.923726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f7c43-fa1e-4240-9301-1f090421fff8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.925932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ed38f-5bfb-4ce9-ab3b-951fe4e3ad52 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e11f7c43-fa1e-4240-9301-1f090421fff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:25.936074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:25.936940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:25.938139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:25.938139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:25.938139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:25.938139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:25.938139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:25.938139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:25.938139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:25.938139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:25.938139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 252/514] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:25.938890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7c43-fa1e-4240-9301-1f090421fff8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.964770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aae962-10d8-49fd-94c3-87ec68fcdab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:25.965315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aae962-10d8-49fd-94c3-87ec68fcdab4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:25.965591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aae962-10d8-49fd-94c3-87ec68fcdab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:25.965858 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aae962-10d8-49fd-94c3-87ec68fcdab4 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:25.966568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aae962-10d8-49fd-94c3-87ec68fcdab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:25.967030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aae962-10d8-49fd-94c3-87ec68fcdab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:25.968394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aae962-10d8-49fd-94c3-87ec68fcdab4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:25.979466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 263/515] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:25.980246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aae962-10d8-49fd-94c3-87ec68fcdab4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:25.996553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:26.021828 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:26.041589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7c43-fa1e-4240-9301-1f090421fff8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.076806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8127f22-35a8-440e-988b-5bed1d540fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.077412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8127f22-35a8-440e-988b-5bed1d540fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.077689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8127f22-35a8-440e-988b-5bed1d540fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.077954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8127f22-35a8-440e-988b-5bed1d540fd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.078665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8127f22-35a8-440e-988b-5bed1d540fd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:26.078966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8127f22-35a8-440e-988b-5bed1d540fd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.080325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8127f22-35a8-440e-988b-5bed1d540fd0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.085288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aae962-10d8-49fd-94c3-87ec68fcdab4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.092385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 253/516] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:26.093182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8127f22-35a8-440e-988b-5bed1d540fd0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.120203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f31fc-2d61-4d39-8d01-d161a94f68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.120826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f31fc-2d61-4d39-8d01-d161a94f68d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.121107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f31fc-2d61-4d39-8d01-d161a94f68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.121384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f31fc-2d61-4d39-8d01-d161a94f68d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.122072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f31fc-2d61-4d39-8d01-d161a94f68d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:26.122374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f31fc-2d61-4d39-8d01-d161a94f68d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.123946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f31fc-2d61-4d39-8d01-d161a94f68d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.136176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 264/517] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:26.136958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f31fc-2d61-4d39-8d01-d161a94f68d2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.202758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8127f22-35a8-440e-988b-5bed1d540fd0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.213844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:26.239924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f905e9-ec6f-4f7d-9e80-b4cdc989c3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.240494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f905e9-ec6f-4f7d-9e80-b4cdc989c3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.240696 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f31fc-2d61-4d39-8d01-d161a94f68d2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.240791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f905e9-ec6f-4f7d-9e80-b4cdc989c3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.241039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f905e9-ec6f-4f7d-9e80-b4cdc989c3d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.249539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:26.252120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f905e9-ec6f-4f7d-9e80-b4cdc989c3d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.253664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f905e9-ec6f-4f7d-9e80-b4cdc989c3d6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b4f905e9-ec6f-4f7d-9e80-b4cdc989c3d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:26.268897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:26.269821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:26.271003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:26.271003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:26.271003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:26.271003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:26.271003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 254/518] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:26.271003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f905e9-ec6f-4f7d-9e80-b4cdc989c3d6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.277656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a72081-d751-4a50-b7fe-fad62623b61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.278313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a72081-d751-4a50-b7fe-fad62623b61b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.278624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a72081-d751-4a50-b7fe-fad62623b61b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.278899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a72081-d751-4a50-b7fe-fad62623b61b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.280294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a72081-d751-4a50-b7fe-fad62623b61b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:26.280598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a72081-d751-4a50-b7fe-fad62623b61b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.282025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a72081-d751-4a50-b7fe-fad62623b61b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.300934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 265/519] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:26.301563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a72081-d751-4a50-b7fe-fad62623b61b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.375676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f905e9-ec6f-4f7d-9e80-b4cdc989c3d6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.400076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a72081-d751-4a50-b7fe-fad62623b61b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.410329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b9dec-e215-4246-a87a-a42f1b8f6589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.410995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b9dec-e215-4246-a87a-a42f1b8f6589 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.411283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b9dec-e215-4246-a87a-a42f1b8f6589 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.411549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b9dec-e215-4246-a87a-a42f1b8f6589 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.418917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b9dec-e215-4246-a87a-a42f1b8f6589 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.421051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b9dec-e215-4246-a87a-a42f1b8f6589 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-df6b9dec-e215-4246-a87a-a42f1b8f6589 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:26.432216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:26.433078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:26.433967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 255/520] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:26.434573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b9dec-e215-4246-a87a-a42f1b8f6589 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.437857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec4a5a-2595-4073-acb9-987af0a41d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.439085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec4a5a-2595-4073-acb9-987af0a41d0f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.439374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec4a5a-2595-4073-acb9-987af0a41d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.439648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec4a5a-2595-4073-acb9-987af0a41d0f None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.440334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec4a5a-2595-4073-acb9-987af0a41d0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.441778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec4a5a-2595-4073-acb9-987af0a41d0f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.449880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:26.463962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 266/521] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:45:26 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:26.464805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec4a5a-2595-4073-acb9-987af0a41d0f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.495956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:26.554051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b9dec-e215-4246-a87a-a42f1b8f6589 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.580967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec4a5a-2595-4073-acb9-987af0a41d0f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.590204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6951bb2-9ca5-4b24-80d8-5da894b63708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.590880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6951bb2-9ca5-4b24-80d8-5da894b63708 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.591159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6951bb2-9ca5-4b24-80d8-5da894b63708 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.591428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6951bb2-9ca5-4b24-80d8-5da894b63708 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.600676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6951bb2-9ca5-4b24-80d8-5da894b63708 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.602158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6951bb2-9ca5-4b24-80d8-5da894b63708 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f6951bb2-9ca5-4b24-80d8-5da894b63708 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:26.614242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:26.615221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 256/522] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:26.616891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6951bb2-9ca5-4b24-80d8-5da894b63708 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.625338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e5792-29aa-4bc6-8456-77ba72e13d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.625976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e5792-29aa-4bc6-8456-77ba72e13d0d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.626252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e5792-29aa-4bc6-8456-77ba72e13d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.626551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e5792-29aa-4bc6-8456-77ba72e13d0d None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.627256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e5792-29aa-4bc6-8456-77ba72e13d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:26.627562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e5792-29aa-4bc6-8456-77ba72e13d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.629658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e5792-29aa-4bc6-8456-77ba72e13d0d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.642989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 267/523] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:26.643770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e5792-29aa-4bc6-8456-77ba72e13d0d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.708494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:26.725303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6951bb2-9ca5-4b24-80d8-5da894b63708 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.740099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:26.764820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e5792-29aa-4bc6-8456-77ba72e13d0d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.766492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a9546-dc4a-4354-b693-20fea6a85da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.767398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a9546-dc4a-4354-b693-20fea6a85da0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.767778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a9546-dc4a-4354-b693-20fea6a85da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.768130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a9546-dc4a-4354-b693-20fea6a85da0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.768984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a9546-dc4a-4354-b693-20fea6a85da0 None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=a1dc3ad467d9496cb3e0086a0cd8215e) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:26.769356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a9546-dc4a-4354-b693-20fea6a85da0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.770956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a9546-dc4a-4354-b693-20fea6a85da0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.786473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 257/524] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:26 2022] GET /identity/v3/role_assignments?user.id=a1dc3ad467d9496cb3e0086a0cd8215e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:26.786473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a9546-dc4a-4354-b693-20fea6a85da0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.817227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87cd5-c0c6-418d-9542-a1c6dceeec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.818046 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87cd5-c0c6-418d-9542-a1c6dceeec43 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.818794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87cd5-c0c6-418d-9542-a1c6dceeec43 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.819085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87cd5-c0c6-418d-9542-a1c6dceeec43 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.820216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e87cd5-c0c6-418d-9542-a1c6dceeec43 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:26.820536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e87cd5-c0c6-418d-9542-a1c6dceeec43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.822917 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e87cd5-c0c6-418d-9542-a1c6dceeec43 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.848161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 268/525] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Jul 30 04:45:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:26.853714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87cd5-c0c6-418d-9542-a1c6dceeec43 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:26.925542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a9546-dc4a-4354-b693-20fea6a85da0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:26.974765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650bc0ce-9238-4883-bf74-8f75329c5ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:26.975400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650bc0ce-9238-4883-bf74-8f75329c5ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:26.975941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650bc0ce-9238-4883-bf74-8f75329c5ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:26.976223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650bc0ce-9238-4883-bf74-8f75329c5ad9 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:26.977604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650bc0ce-9238-4883-bf74-8f75329c5ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:26.978039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650bc0ce-9238-4883-bf74-8f75329c5ad9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:26.980209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650bc0ce-9238-4883-bf74-8f75329c5ad9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:26.986406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87cd5-c0c6-418d-9542-a1c6dceeec43 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:27.000038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:27.004921 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 258/526] 10.222.0.20 () {56 vars in 1139 bytes} [Sat Jul 30 04:45:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:27.006845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650bc0ce-9238-4883-bf74-8f75329c5ad9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:27.038182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1bfc7-182d-4a03-a7a3-bcba555bd650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:27.039036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1bfc7-182d-4a03-a7a3-bcba555bd650 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:27.039416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1bfc7-182d-4a03-a7a3-bcba555bd650 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:27.039789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1bfc7-182d-4a03-a7a3-bcba555bd650 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:27.047255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1bfc7-182d-4a03-a7a3-bcba555bd650 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:27.048826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1bfc7-182d-4a03-a7a3-bcba555bd650 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:27.054823 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f3e1bfc7-182d-4a03-a7a3-bcba555bd650 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:27.062429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:27.063534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:27.064609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:27.064609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:27.064609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:27.064609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:27.064609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:27.064609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:27.064609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:27.064609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:27.070697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 269/527] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:27.071656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1bfc7-182d-4a03-a7a3-bcba555bd650 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:27.146787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650bc0ce-9238-4883-bf74-8f75329c5ad9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:27.195488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1bfc7-182d-4a03-a7a3-bcba555bd650 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:27.199695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10763e57-709a-4ba9-8daf-5c5c2192499c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:27.200499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10763e57-709a-4ba9-8daf-5c5c2192499c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:27.200929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10763e57-709a-4ba9-8daf-5c5c2192499c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:27.201942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10763e57-709a-4ba9-8daf-5c5c2192499c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:27.235042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd70c-01f9-4bd6-88a9-fbabe08d2613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:27.235874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd70c-01f9-4bd6-88a9-fbabe08d2613 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:27.236253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd70c-01f9-4bd6-88a9-fbabe08d2613 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:27.236625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbd70c-01f9-4bd6-88a9-fbabe08d2613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:27.275309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:27.296823 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10763e57-709a-4ba9-8daf-5c5c2192499c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:27.298871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10763e57-709a-4ba9-8daf-5c5c2192499c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:27.316945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd70c-01f9-4bd6-88a9-fbabe08d2613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:27.318557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbd70c-01f9-4bd6-88a9-fbabe08d2613 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:27.335233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:27.407345 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 259/528] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:27.408611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10763e57-709a-4ba9-8daf-5c5c2192499c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:27.458592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 270/529] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:27.459698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd70c-01f9-4bd6-88a9-fbabe08d2613 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:27.536852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10763e57-709a-4ba9-8daf-5c5c2192499c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:27.538036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:27.574656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67307ed8-3fc4-43c2-a14e-771dce7d121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:27.575513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67307ed8-3fc4-43c2-a14e-771dce7d121a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:27.575940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67307ed8-3fc4-43c2-a14e-771dce7d121a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:27.576396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67307ed8-3fc4-43c2-a14e-771dce7d121a None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:27.577293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67307ed8-3fc4-43c2-a14e-771dce7d121a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:27.577771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67307ed8-3fc4-43c2-a14e-771dce7d121a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:27.579404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67307ed8-3fc4-43c2-a14e-771dce7d121a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:27.581492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:27.583899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbd70c-01f9-4bd6-88a9-fbabe08d2613 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:27.600592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 260/530] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:27.602304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67307ed8-3fc4-43c2-a14e-771dce7d121a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:27.627170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaaff7-282d-44d5-ac57-323aecf13115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:27.627909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaaff7-282d-44d5-ac57-323aecf13115 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:27.628256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaaff7-282d-44d5-ac57-323aecf13115 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:27.628601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaaff7-282d-44d5-ac57-323aecf13115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:27.641221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccaaff7-282d-44d5-ac57-323aecf13115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:27.642876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccaaff7-282d-44d5-ac57-323aecf13115 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7ccaaff7-282d-44d5-ac57-323aecf13115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:27.660045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:27.660987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:27.661866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:27.661866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:27.661866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:27.661866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:27.662738 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 271/531] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:27.663613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaaff7-282d-44d5-ac57-323aecf13115 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:27.718959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67307ed8-3fc4-43c2-a14e-771dce7d121a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:27.756669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f999ab-a2d0-4c2d-b6f3-de818ee4c388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:27.757312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f999ab-a2d0-4c2d-b6f3-de818ee4c388 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:27.757588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f999ab-a2d0-4c2d-b6f3-de818ee4c388 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:27.757859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f999ab-a2d0-4c2d-b6f3-de818ee4c388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:27.769877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f999ab-a2d0-4c2d-b6f3-de818ee4c388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:27.771357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f999ab-a2d0-4c2d-b6f3-de818ee4c388 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:27.775614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaaff7-282d-44d5-ac57-323aecf13115 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-90f999ab-a2d0-4c2d-b6f3-de818ee4c388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:27.787793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:27.789768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:27.791976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:27.791976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:27.791976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:27.791976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:27.793699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 261/532] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:27.798552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f999ab-a2d0-4c2d-b6f3-de818ee4c388 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:27.798762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:27.819913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb26b6-0fee-40e9-be18-c4a7957bac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:27.820904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb26b6-0fee-40e9-be18-c4a7957bac26 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:27.821182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb26b6-0fee-40e9-be18-c4a7957bac26 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:27.821451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb26b6-0fee-40e9-be18-c4a7957bac26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:27.822230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb26b6-0fee-40e9-be18-c4a7957bac26 None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=548acf58bf9140ec9f1ba126e10deb45) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:27.822562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb26b6-0fee-40e9-be18-c4a7957bac26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:27.824804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb26b6-0fee-40e9-be18-c4a7957bac26 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:27.833113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 272/533] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:27 2022] GET /identity/v3/role_assignments?user.id=548acf58bf9140ec9f1ba126e10deb45&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:27.834304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb26b6-0fee-40e9-be18-c4a7957bac26 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:27.852449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:27.918793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f999ab-a2d0-4c2d-b6f3-de818ee4c388 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:27.957518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479dbaf0-8de7-4a5e-b2b9-dac7dc703783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:27.958356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479dbaf0-8de7-4a5e-b2b9-dac7dc703783 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:27.958766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479dbaf0-8de7-4a5e-b2b9-dac7dc703783 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:27.959129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479dbaf0-8de7-4a5e-b2b9-dac7dc703783 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:27.960025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479dbaf0-8de7-4a5e-b2b9-dac7dc703783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:27.960419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479dbaf0-8de7-4a5e-b2b9-dac7dc703783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:27.962039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479dbaf0-8de7-4a5e-b2b9-dac7dc703783 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:27.978638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 262/534] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:27.980481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb26b6-0fee-40e9-be18-c4a7957bac26 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:27.985036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479dbaf0-8de7-4a5e-b2b9-dac7dc703783 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:28.028446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b690b-1d14-40b1-81d9-65cef0b45a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:28.030111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b690b-1d14-40b1-81d9-65cef0b45a72 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:28.030690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b690b-1d14-40b1-81d9-65cef0b45a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:28.030967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b690b-1d14-40b1-81d9-65cef0b45a72 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:28.031746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b690b-1d14-40b1-81d9-65cef0b45a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:28.032172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b690b-1d14-40b1-81d9-65cef0b45a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:28.033745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b690b-1d14-40b1-81d9-65cef0b45a72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:28.051390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 273/535] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:28.056393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b690b-1d14-40b1-81d9-65cef0b45a72 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:28.127735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479dbaf0-8de7-4a5e-b2b9-dac7dc703783 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:28.136296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:28.151204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:28.196330 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce47c-8e40-4b91-bca9-2bf882e1ea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:28.197043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce47c-8e40-4b91-bca9-2bf882e1ea17 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:28.197319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce47c-8e40-4b91-bca9-2bf882e1ea17 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:28.197586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ce47c-8e40-4b91-bca9-2bf882e1ea17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:28.212865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce47c-8e40-4b91-bca9-2bf882e1ea17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:28.214337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ce47c-8e40-4b91-bca9-2bf882e1ea17 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b31ce47c-8e40-4b91-bca9-2bf882e1ea17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:28.245641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:28.249417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:28.251530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:28.251530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:28.251530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:28.251530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:28.251530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:28.251530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:28.251530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:28.251530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:28.251530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 263/536] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:28.260540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce47c-8e40-4b91-bca9-2bf882e1ea17 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:28.260796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b690b-1d14-40b1-81d9-65cef0b45a72 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:28.326806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d5a5c-40ec-4ee1-95db-3d1feb12d053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:28.327801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d5a5c-40ec-4ee1-95db-3d1feb12d053 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:28.328290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d5a5c-40ec-4ee1-95db-3d1feb12d053 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:28.328766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d5a5c-40ec-4ee1-95db-3d1feb12d053 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:28.339722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d5a5c-40ec-4ee1-95db-3d1feb12d053 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:28.342784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d5a5c-40ec-4ee1-95db-3d1feb12d053 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7c4d5a5c-40ec-4ee1-95db-3d1feb12d053 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:28.365346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:28.366234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:28.367129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:28.367129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:28.367129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:28.367129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:28.367129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:28.367129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:28.367129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:28.367129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:28.375940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 274/537] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:28.379823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d5a5c-40ec-4ee1-95db-3d1feb12d053 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:28.451675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ce47c-8e40-4b91-bca9-2bf882e1ea17 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:28.521161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4f521-feb8-4146-9601-96bd768581d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:28.522183 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4f521-feb8-4146-9601-96bd768581d9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:28.522708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4f521-feb8-4146-9601-96bd768581d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:28.523307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4f521-feb8-4146-9601-96bd768581d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:28.525571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4f521-feb8-4146-9601-96bd768581d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:28.526084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4f521-feb8-4146-9601-96bd768581d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:28.528029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d4f521-feb8-4146-9601-96bd768581d9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:28.540755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 264/538] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:28.547195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4f521-feb8-4146-9601-96bd768581d9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:28.576501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d5a5c-40ec-4ee1-95db-3d1feb12d053 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:28.579609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:28.602797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:28.639107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f336e2-c85c-45df-b199-1b8bbc3de1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:28.640139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f336e2-c85c-45df-b199-1b8bbc3de1fe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:28.640627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f336e2-c85c-45df-b199-1b8bbc3de1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:28.641103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f336e2-c85c-45df-b199-1b8bbc3de1fe None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:28.643039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f336e2-c85c-45df-b199-1b8bbc3de1fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:28.645169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f336e2-c85c-45df-b199-1b8bbc3de1fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:28.648085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f336e2-c85c-45df-b199-1b8bbc3de1fe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:28.671201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 275/539] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:28.673312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8955bd8-308d-413a-bce5-e1fc6d7b30fc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:28.673605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8955bd8-308d-413a-bce5-e1fc6d7b30fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:28.673885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8955bd8-308d-413a-bce5-e1fc6d7b30fc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:28.675425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 276/540] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:28.676586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8955bd8-308d-413a-bce5-e1fc6d7b30fc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:28.706994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4f521-feb8-4146-9601-96bd768581d9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:28.747600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c0019-6197-4dfb-87c5-1733d50d9037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:28.748659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c0019-6197-4dfb-87c5-1733d50d9037 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:28.749190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c0019-6197-4dfb-87c5-1733d50d9037 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:28.749511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c0019-6197-4dfb-87c5-1733d50d9037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:28.750464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c0019-6197-4dfb-87c5-1733d50d9037 None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=a0d2552bd0e244dea55eb1eecf471fef) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:28.750786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c0019-6197-4dfb-87c5-1733d50d9037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:28.752873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c0019-6197-4dfb-87c5-1733d50d9037 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:28.760029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 265/541] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:28 2022] GET /identity/v3/role_assignments?user.id=a0d2552bd0e244dea55eb1eecf471fef&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:28.761497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3b3f2-0b2c-4185-8933-515cbc00592d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:28.761777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3b3f2-0b2c-4185-8933-515cbc00592d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:28.762048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3b3f2-0b2c-4185-8933-515cbc00592d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:28.787022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f0e3b3f2-0b2c-4185-8933-515cbc00592d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:28.788338 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f0e3b3f2-0b2c-4185-8933-515cbc00592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:28.828802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8955bd8-308d-413a-bce5-e1fc6d7b30fc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:28.910933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb2d34-9fae-46fe-9f39-2b1fe145cc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:28.911487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb2d34-9fae-46fe-9f39-2b1fe145cc16 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:28.911788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb2d34-9fae-46fe-9f39-2b1fe145cc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:28.912028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb2d34-9fae-46fe-9f39-2b1fe145cc16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:28.916891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb2d34-9fae-46fe-9f39-2b1fe145cc16 None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=426be56424504e958402d785a068d2f8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:28.917214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb2d34-9fae-46fe-9f39-2b1fe145cc16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:28.918695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb2d34-9fae-46fe-9f39-2b1fe145cc16 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:28.929312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 277/542] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:28 2022] GET /identity/v3/role_assignments?user.id=426be56424504e958402d785a068d2f8&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:28.937139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 266/543] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:28.960882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb2d34-9fae-46fe-9f39-2b1fe145cc16 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:28.998914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:29.085121 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb2d34-9fae-46fe-9f39-2b1fe145cc16 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:29.125158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a223748-d392-426e-9a36-3e56576c4cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:29.125800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a223748-d392-426e-9a36-3e56576c4cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:29.126076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a223748-d392-426e-9a36-3e56576c4cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:29.126346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a223748-d392-426e-9a36-3e56576c4cc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:29.137212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a223748-d392-426e-9a36-3e56576c4cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6a223748-d392-426e-9a36-3e56576c4cc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:29.143028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:29.143953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:29.145198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 278/544] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:29.150491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3b3f2-0b2c-4185-8933-515cbc00592d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:29.174975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:29.205215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09b700-aa64-4b2a-8c1d-e288e67d4cf2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:29.205596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09b700-aa64-4b2a-8c1d-e288e67d4cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:29.205758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09b700-aa64-4b2a-8c1d-e288e67d4cf2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:29.207570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 279/545] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:29.216324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89edfae-f11f-493a-912f-d68de6f25f2a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:29.217530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89edfae-f11f-493a-912f-d68de6f25f2a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:29.217801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89edfae-f11f-493a-912f-d68de6f25f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:29.241692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c89edfae-f11f-493a-912f-d68de6f25f2a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:29.243441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c89edfae-f11f-493a-912f-d68de6f25f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:29.270581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3b3f2-0b2c-4185-8933-515cbc00592d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:29.327337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 280/546] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 30 04:45:29.353085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba0e17-2aed-4ebc-8a4a-221450cdb139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:29.353705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba0e17-2aed-4ebc-8a4a-221450cdb139 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:29.353979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba0e17-2aed-4ebc-8a4a-221450cdb139 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:29.354246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba0e17-2aed-4ebc-8a4a-221450cdb139 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:29.355039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba0e17-2aed-4ebc-8a4a-221450cdb139 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:29.355338 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba0e17-2aed-4ebc-8a4a-221450cdb139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:29.356748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba0e17-2aed-4ebc-8a4a-221450cdb139 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:29.366349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 267/547] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:29.371583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89edfae-f11f-493a-912f-d68de6f25f2a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:29.383119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba0e17-2aed-4ebc-8a4a-221450cdb139 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:29.434467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:29.475305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:29.495534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba0e17-2aed-4ebc-8a4a-221450cdb139 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:29.534361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d959acd-54f5-4a95-87da-d32ec48877a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:29.535313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d959acd-54f5-4a95-87da-d32ec48877a3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:29.535591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d959acd-54f5-4a95-87da-d32ec48877a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:29.535954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d959acd-54f5-4a95-87da-d32ec48877a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:29.550164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d959acd-54f5-4a95-87da-d32ec48877a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:29.551762 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d959acd-54f5-4a95-87da-d32ec48877a3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3d959acd-54f5-4a95-87da-d32ec48877a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:29.569115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:29.570000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:29.571295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:29.571295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:29.571295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:29.571295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:29.571295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 268/548] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:29.577007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d959acd-54f5-4a95-87da-d32ec48877a3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:29.684563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89edfae-f11f-493a-912f-d68de6f25f2a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:29.694517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d959acd-54f5-4a95-87da-d32ec48877a3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:29.732985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741981f-141e-4de6-ae1e-7dbb1d8168c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:29.733602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741981f-141e-4de6-ae1e-7dbb1d8168c0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:29.733922 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741981f-141e-4de6-ae1e-7dbb1d8168c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:29.734192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741981f-141e-4de6-ae1e-7dbb1d8168c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:29.735046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741981f-141e-4de6-ae1e-7dbb1d8168c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:29.735644 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741981f-141e-4de6-ae1e-7dbb1d8168c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:29.737021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741981f-141e-4de6-ae1e-7dbb1d8168c0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:29.746888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:29.754903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 269/549] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:29.756324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741981f-141e-4de6-ae1e-7dbb1d8168c0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:29.810628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bffd5-ef60-49d9-bb46-e2b68c5af070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:29.812163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bffd5-ef60-49d9-bb46-e2b68c5af070 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:29.812444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bffd5-ef60-49d9-bb46-e2b68c5af070 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:29.812712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bffd5-ef60-49d9-bb46-e2b68c5af070 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:29.859823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bffd5-ef60-49d9-bb46-e2b68c5af070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-317bffd5-ef60-49d9-bb46-e2b68c5af070 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:29.870724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:29.872093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:29.873768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:29.875392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 281/550] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 504 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:29.882925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741981f-141e-4de6-ae1e-7dbb1d8168c0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:29.887641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bffd5-ef60-49d9-bb46-e2b68c5af070 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:29.922974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4030c47-40c9-4872-8708-bfe92a2af672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:29.923772 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4030c47-40c9-4872-8708-bfe92a2af672 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:29.924058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4030c47-40c9-4872-8708-bfe92a2af672 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:29.924478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4030c47-40c9-4872-8708-bfe92a2af672 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:29.932351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4030c47-40c9-4872-8708-bfe92a2af672 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:29.934011 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4030c47-40c9-4872-8708-bfe92a2af672 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c4030c47-40c9-4872-8708-bfe92a2af672 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:29.948309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:29.949253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:29.950790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 270/551] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:29.956185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4030c47-40c9-4872-8708-bfe92a2af672 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:30.034122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:30.053709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bffd5-ef60-49d9-bb46-e2b68c5af070 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:30.100439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fc7c9-3305-4ddb-8b27-d2000855e7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:30.101416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fc7c9-3305-4ddb-8b27-d2000855e7ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:30.101902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fc7c9-3305-4ddb-8b27-d2000855e7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:30.102377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fc7c9-3305-4ddb-8b27-d2000855e7ac None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:30.103344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fc7c9-3305-4ddb-8b27-d2000855e7ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:30.103871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fc7c9-3305-4ddb-8b27-d2000855e7ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:30.106879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fc7c9-3305-4ddb-8b27-d2000855e7ac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:30.121105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 282/552] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:30.126243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fc7c9-3305-4ddb-8b27-d2000855e7ac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:30.130042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:30.194386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4030c47-40c9-4872-8708-bfe92a2af672 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:30.232440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd339d69-45ca-4a47-b7ca-c5c9d55b2176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:30.233095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd339d69-45ca-4a47-b7ca-c5c9d55b2176 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:30.233426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd339d69-45ca-4a47-b7ca-c5c9d55b2176 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:30.233707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd339d69-45ca-4a47-b7ca-c5c9d55b2176 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:30.234778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd339d69-45ca-4a47-b7ca-c5c9d55b2176 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:30.234982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd339d69-45ca-4a47-b7ca-c5c9d55b2176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:30.236427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd339d69-45ca-4a47-b7ca-c5c9d55b2176 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:30.253314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fc7c9-3305-4ddb-8b27-d2000855e7ac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:30.254914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 271/553] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Jul 30 04:45:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:30.256976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54e541-acaa-401b-9197-b2a3de17474f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:30.256976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54e541-acaa-401b-9197-b2a3de17474f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:30.256976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54e541-acaa-401b-9197-b2a3de17474f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:30.258499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 272/554] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:30.259864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54e541-acaa-401b-9197-b2a3de17474f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:30.297856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5eb73d-1ba5-47bb-b718-92d390fb52eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:30.299576 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eb73d-1ba5-47bb-b718-92d390fb52eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:30.300034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eb73d-1ba5-47bb-b718-92d390fb52eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:30.300463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eb73d-1ba5-47bb-b718-92d390fb52eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:30.312032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:30.315189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eb73d-1ba5-47bb-b718-92d390fb52eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:30.317638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eb73d-1ba5-47bb-b718-92d390fb52eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cc5eb73d-1ba5-47bb-b718-92d390fb52eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:30.337203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:30.338751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:30.339834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:30.339834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:30.339834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:30.339834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:30.343260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 283/555] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:30.344667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597cc30-e8fc-417e-a071-167a0ba009dc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:30.345085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597cc30-e8fc-417e-a071-167a0ba009dc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:30.345493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597cc30-e8fc-417e-a071-167a0ba009dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:30.367769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b597cc30-e8fc-417e-a071-167a0ba009dc None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:30.369202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b597cc30-e8fc-417e-a071-167a0ba009dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:30.379639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54e541-acaa-401b-9197-b2a3de17474f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:30.396034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:30.423007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c28d1-8e83-451e-899c-c771d88e0b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:30.423983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c28d1-8e83-451e-899c-c771d88e0b77 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:30.424417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c28d1-8e83-451e-899c-c771d88e0b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:30.425180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c28d1-8e83-451e-899c-c771d88e0b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:30.456633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 284/556] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:30.457782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597cc30-e8fc-417e-a071-167a0ba009dc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:30.527497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c28d1-8e83-451e-899c-c771d88e0b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:30.529466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c28d1-8e83-451e-899c-c771d88e0b77 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:30.605664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597cc30-e8fc-417e-a071-167a0ba009dc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:30.638046 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:30.640883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 273/557] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:30.641882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c28d1-8e83-451e-899c-c771d88e0b77 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:30.653420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414a9b5-54f9-453c-aed1-9df962df29b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:30.655049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414a9b5-54f9-453c-aed1-9df962df29b9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:30.655337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414a9b5-54f9-453c-aed1-9df962df29b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:30.655613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414a9b5-54f9-453c-aed1-9df962df29b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:30.656740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414a9b5-54f9-453c-aed1-9df962df29b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:30.657188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414a9b5-54f9-453c-aed1-9df962df29b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:30.659026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:30.659869 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414a9b5-54f9-453c-aed1-9df962df29b9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:30.678529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 285/558] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:30.679411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414a9b5-54f9-453c-aed1-9df962df29b9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:30.769358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c28d1-8e83-451e-899c-c771d88e0b77 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:30.798839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414a9b5-54f9-453c-aed1-9df962df29b9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:30.815614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6de4a-278c-4760-bd8e-040364f17504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:30.816267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6de4a-278c-4760-bd8e-040364f17504 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:30.816700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6de4a-278c-4760-bd8e-040364f17504 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:30.816981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6de4a-278c-4760-bd8e-040364f17504 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:30.830904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6de4a-278c-4760-bd8e-040364f17504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:30.835448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac83fd-87fd-460f-95e8-f1d2dd696209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3db6de4a-278c-4760-bd8e-040364f17504 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:30.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:30.838114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:30.840188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:30.842034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac83fd-87fd-460f-95e8-f1d2dd696209 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:30.842034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac83fd-87fd-460f-95e8-f1d2dd696209 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:30.842034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac83fd-87fd-460f-95e8-f1d2dd696209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:30.842334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 274/559] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:30.842334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6de4a-278c-4760-bd8e-040364f17504 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:30.859477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac83fd-87fd-460f-95e8-f1d2dd696209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:30.861715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac83fd-87fd-460f-95e8-f1d2dd696209 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-27ac83fd-87fd-460f-95e8-f1d2dd696209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:30.880640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:30.882355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:30.883979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:30.883979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:30.883979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:30.883979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:30.883979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 286/560] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:30.883979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac83fd-87fd-460f-95e8-f1d2dd696209 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:30.926774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:30.936195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:30.971571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6de4a-278c-4760-bd8e-040364f17504 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.011734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac83fd-87fd-460f-95e8-f1d2dd696209 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.024366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c47c3-b8e5-43fd-bf79-7764b9f1b321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.025475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c47c3-b8e5-43fd-bf79-7764b9f1b321 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.025889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c47c3-b8e5-43fd-bf79-7764b9f1b321 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.026178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c47c3-b8e5-43fd-bf79-7764b9f1b321 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.036824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c47c3-b8e5-43fd-bf79-7764b9f1b321 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.038280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c47c3-b8e5-43fd-bf79-7764b9f1b321 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-107c47c3-b8e5-43fd-bf79-7764b9f1b321 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:31.051139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:31.052108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 275/561] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Jul 30 04:45:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:31.053063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c47c3-b8e5-43fd-bf79-7764b9f1b321 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.055188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757920-7426-4979-bf52-b86f5db7e76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.055707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757920-7426-4979-bf52-b86f5db7e76d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.056413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757920-7426-4979-bf52-b86f5db7e76d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.056413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757920-7426-4979-bf52-b86f5db7e76d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.057558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757920-7426-4979-bf52-b86f5db7e76d None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:31.057797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757920-7426-4979-bf52-b86f5db7e76d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.059315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757920-7426-4979-bf52-b86f5db7e76d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.072854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 287/562] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:31.072854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c75ef-de36-4b52-a872-f0ab39a6076e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.072854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c75ef-de36-4b52-a872-f0ab39a6076e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.072854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c75ef-de36-4b52-a872-f0ab39a6076e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.074634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 288/563] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:31.076624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c75ef-de36-4b52-a872-f0ab39a6076e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.171625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c47c3-b8e5-43fd-bf79-7764b9f1b321 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.197954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:31.204625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:31.205978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c75ef-de36-4b52-a872-f0ab39a6076e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.211105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38281c8-bfd0-49cb-9b11-791620a1ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.211728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38281c8-bfd0-49cb-9b11-791620a1ab6f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.212006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38281c8-bfd0-49cb-9b11-791620a1ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.212275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38281c8-bfd0-49cb-9b11-791620a1ab6f None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.213032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38281c8-bfd0-49cb-9b11-791620a1ab6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:31.213332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38281c8-bfd0-49cb-9b11-791620a1ab6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.215196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38281c8-bfd0-49cb-9b11-791620a1ab6f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.232866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 276/564] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:31.232866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38281c8-bfd0-49cb-9b11-791620a1ab6f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.249982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836cdb6-8f8e-407b-8c88-22adf9913ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.250637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836cdb6-8f8e-407b-8c88-22adf9913ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.250910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836cdb6-8f8e-407b-8c88-22adf9913ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.251493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836cdb6-8f8e-407b-8c88-22adf9913ed7 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.251966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836cdb6-8f8e-407b-8c88-22adf9913ed7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:31.252275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836cdb6-8f8e-407b-8c88-22adf9913ed7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.253705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836cdb6-8f8e-407b-8c88-22adf9913ed7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.272634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 289/565] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:31.277326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f320b-238e-4d79-a7b6-0d8cf93b4365 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.277326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f320b-238e-4d79-a7b6-0d8cf93b4365 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.277326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f320b-238e-4d79-a7b6-0d8cf93b4365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.293838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e71f320b-238e-4d79-a7b6-0d8cf93b4365 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:31.295209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e71f320b-238e-4d79-a7b6-0d8cf93b4365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:31.339517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38281c8-bfd0-49cb-9b11-791620a1ab6f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.366752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 290/566] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:31.366752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f320b-238e-4d79-a7b6-0d8cf93b4365 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.379067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984456-d863-4b88-bff2-7577204ecbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.379067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984456-d863-4b88-bff2-7577204ecbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.379067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984456-d863-4b88-bff2-7577204ecbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.379067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984456-d863-4b88-bff2-7577204ecbf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.392857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984456-d863-4b88-bff2-7577204ecbf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.394317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984456-d863-4b88-bff2-7577204ecbf7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0a984456-d863-4b88-bff2-7577204ecbf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:31.409125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:31.410050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:31.411002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:31.411002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:31.411002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:31.411002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:31.411002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 277/567] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:31.411290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984456-d863-4b88-bff2-7577204ecbf7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.457029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:31.464174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:31.480735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f320b-238e-4d79-a7b6-0d8cf93b4365 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.511824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984456-d863-4b88-bff2-7577204ecbf7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.515580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e8998-1bd4-4f6a-a725-0acf045e6ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.516224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e8998-1bd4-4f6a-a725-0acf045e6ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.516505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e8998-1bd4-4f6a-a725-0acf045e6ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.516778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e8998-1bd4-4f6a-a725-0acf045e6ee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.523523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e8998-1bd4-4f6a-a725-0acf045e6ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.524982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e8998-1bd4-4f6a-a725-0acf045e6ee8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c49e8998-1bd4-4f6a-a725-0acf045e6ee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:31.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:31.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:31.538993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:31.538993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:31.538993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:31.538993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:31.538993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:31.538993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:31.538993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:31.538993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:31.538993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 291/568] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:31.539769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067abf3-4cb3-4556-8a15-da130c9cd5af None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.539945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067abf3-4cb3-4556-8a15-da130c9cd5af None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.540222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067abf3-4cb3-4556-8a15-da130c9cd5af None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.541465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 292/569] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:31.542152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067abf3-4cb3-4556-8a15-da130c9cd5af None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.550223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8109b76-a28a-4bbf-ab10-860fa503b331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.550957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8109b76-a28a-4bbf-ab10-860fa503b331 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.551235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8109b76-a28a-4bbf-ab10-860fa503b331 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.551505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8109b76-a28a-4bbf-ab10-860fa503b331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.562551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8109b76-a28a-4bbf-ab10-860fa503b331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.564001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8109b76-a28a-4bbf-ab10-860fa503b331 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c8109b76-a28a-4bbf-ab10-860fa503b331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:31.578539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:31.579398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:31.580289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:31.580289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:31.580289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:31.580289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:31.580289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 278/570] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:31.580588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8109b76-a28a-4bbf-ab10-860fa503b331 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.645572 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067abf3-4cb3-4556-8a15-da130c9cd5af None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.675869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:31.681125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5a4bd-f7b4-4657-97a5-f3dae4f6e9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.681762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5a4bd-f7b4-4657-97a5-f3dae4f6e9fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.681862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8109b76-a28a-4bbf-ab10-860fa503b331 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.682039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5a4bd-f7b4-4657-97a5-f3dae4f6e9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.682309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5a4bd-f7b4-4657-97a5-f3dae4f6e9fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.694408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5a4bd-f7b4-4657-97a5-f3dae4f6e9fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-33a5a4bd-f7b4-4657-97a5-f3dae4f6e9fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:31.699249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:31.700179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:31.701109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 293/571] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:31.702011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5a4bd-f7b4-4657-97a5-f3dae4f6e9fb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.705061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:31.719837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390d176-fb21-42b8-86fb-7e80d9e99d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.720452 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390d176-fb21-42b8-86fb-7e80d9e99d4d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.720725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390d176-fb21-42b8-86fb-7e80d9e99d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.720993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390d176-fb21-42b8-86fb-7e80d9e99d4d None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.721673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390d176-fb21-42b8-86fb-7e80d9e99d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:31.721970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390d176-fb21-42b8-86fb-7e80d9e99d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.723345 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390d176-fb21-42b8-86fb-7e80d9e99d4d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.735575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 279/572] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:31.736717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1268a-0dd9-447e-a056-b5b2ca208b7b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.736997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1268a-0dd9-447e-a056-b5b2ca208b7b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.737265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1268a-0dd9-447e-a056-b5b2ca208b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.754945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b7f1268a-0dd9-447e-a056-b5b2ca208b7b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:31.756124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b7f1268a-0dd9-447e-a056-b5b2ca208b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:31.803228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5a4bd-f7b4-4657-97a5-f3dae4f6e9fb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.822465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 280/573] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:31.823284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1268a-0dd9-447e-a056-b5b2ca208b7b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.840334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d576f-022d-461d-bc74-b1afe54f38df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.841068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d576f-022d-461d-bc74-b1afe54f38df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.841353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d576f-022d-461d-bc74-b1afe54f38df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.842188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d576f-022d-461d-bc74-b1afe54f38df None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.842966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d576f-022d-461d-bc74-b1afe54f38df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:31.843271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d576f-022d-461d-bc74-b1afe54f38df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.844708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d576f-022d-461d-bc74-b1afe54f38df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.856320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 294/574] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:31.857153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d576f-022d-461d-bc74-b1afe54f38df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.918615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:31.926370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:31.941880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1268a-0dd9-447e-a056-b5b2ca208b7b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.958648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d576f-022d-461d-bc74-b1afe54f38df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:31.976046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657ed37-d425-4165-a90b-5b3fd88752fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.976664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657ed37-d425-4165-a90b-5b3fd88752fc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.976944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657ed37-d425-4165-a90b-5b3fd88752fc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.977218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657ed37-d425-4165-a90b-5b3fd88752fc None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.977907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657ed37-d425-4165-a90b-5b3fd88752fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:31.978210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657ed37-d425-4165-a90b-5b3fd88752fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.979741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657ed37-d425-4165-a90b-5b3fd88752fc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:31.995110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 281/575] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:31.995110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657ed37-d425-4165-a90b-5b3fd88752fc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:31.995372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d2fb0-cf08-4653-8ce9-9b9027447031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:31.995372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d2fb0-cf08-4653-8ce9-9b9027447031 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:31.995372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d2fb0-cf08-4653-8ce9-9b9027447031 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:31.995372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d2fb0-cf08-4653-8ce9-9b9027447031 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:31.995720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d2fb0-cf08-4653-8ce9-9b9027447031 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:31.995720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d2fb0-cf08-4653-8ce9-9b9027447031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:31.996417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d2fb0-cf08-4653-8ce9-9b9027447031 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.006040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 295/576] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:32.006883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d2fb0-cf08-4653-8ce9-9b9027447031 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.099518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657ed37-d425-4165-a90b-5b3fd88752fc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.109743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d2fb0-cf08-4653-8ce9-9b9027447031 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.132492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d38bc-8766-4cf2-8103-e763148b4c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.133128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d38bc-8766-4cf2-8103-e763148b4c59 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.133404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d38bc-8766-4cf2-8103-e763148b4c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.133671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d38bc-8766-4cf2-8103-e763148b4c59 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.135132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:32.141881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d38bc-8766-4cf2-8103-e763148b4c59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:32.143358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d38bc-8766-4cf2-8103-e763148b4c59 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.144609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82166a-076e-4fad-a073-543cada67c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.145239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82166a-076e-4fad-a073-543cada67c4c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.145513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82166a-076e-4fad-a073-543cada67c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.145779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82166a-076e-4fad-a073-543cada67c4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d71d38bc-8766-4cf2-8103-e763148b4c59 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:32.154726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:32.155861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:32.156835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 282/577] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:32.157427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82166a-076e-4fad-a073-543cada67c4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:32.157507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d38bc-8766-4cf2-8103-e763148b4c59 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0d82166a-076e-4fad-a073-543cada67c4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:32.161361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:32.162380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 296/578] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:32.164618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82166a-076e-4fad-a073-543cada67c4c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.166819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:32.257099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d38bc-8766-4cf2-8103-e763148b4c59 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.266691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82166a-076e-4fad-a073-543cada67c4c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.290194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a54182-659d-4ff9-ac08-91e12a948fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.290837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a54182-659d-4ff9-ac08-91e12a948fec None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.291117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a54182-659d-4ff9-ac08-91e12a948fec None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.291388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a54182-659d-4ff9-ac08-91e12a948fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.292159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a54182-659d-4ff9-ac08-91e12a948fec None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=54bda2af62ee4ee39a9a350b7ce6430e) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:32.292464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a54182-659d-4ff9-ac08-91e12a948fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:32.293880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a54182-659d-4ff9-ac08-91e12a948fec None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.299648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 283/579] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/role_assignments?user.id=54bda2af62ee4ee39a9a350b7ce6430e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:32.299778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777dc6a-d8fc-44ad-9c15-3238f606ae70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.300400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777dc6a-d8fc-44ad-9c15-3238f606ae70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.300504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a54182-659d-4ff9-ac08-91e12a948fec None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.300758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777dc6a-d8fc-44ad-9c15-3238f606ae70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.301108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777dc6a-d8fc-44ad-9c15-3238f606ae70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.308300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777dc6a-d8fc-44ad-9c15-3238f606ae70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:32.309788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777dc6a-d8fc-44ad-9c15-3238f606ae70 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b777dc6a-d8fc-44ad-9c15-3238f606ae70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:32.324039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:32.325031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:32.326549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:32.326549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:32.326549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:32.326549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:32.326549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:32.326549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:32.326549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:32.326549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:32.326549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 297/580] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:32.327056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777dc6a-d8fc-44ad-9c15-3238f606ae70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.383340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:32.384001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:32.421044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a54182-659d-4ff9-ac08-91e12a948fec None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.454271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777dc6a-d8fc-44ad-9c15-3238f606ae70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.459832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcb6ef-c1c9-48e2-b7d8-a145a70d7900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.460475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcb6ef-c1c9-48e2-b7d8-a145a70d7900 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.460753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcb6ef-c1c9-48e2-b7d8-a145a70d7900 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.461023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcb6ef-c1c9-48e2-b7d8-a145a70d7900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.472784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bcb6ef-c1c9-48e2-b7d8-a145a70d7900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:32.474735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bcb6ef-c1c9-48e2-b7d8-a145a70d7900 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.492022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed54384-afa8-40b1-a66d-1feae307d4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d3bcb6ef-c1c9-48e2-b7d8-a145a70d7900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:32.492324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:32.493190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:32.494115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:32.494115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:32.494115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:32.494115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:32.494335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed54384-afa8-40b1-a66d-1feae307d4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.494335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed54384-afa8-40b1-a66d-1feae307d4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.494335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed54384-afa8-40b1-a66d-1feae307d4d8 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.494335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed54384-afa8-40b1-a66d-1feae307d4d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:32.494335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed54384-afa8-40b1-a66d-1feae307d4d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:32.495764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed54384-afa8-40b1-a66d-1feae307d4d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.498938 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 284/581] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:32.499654 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcb6ef-c1c9-48e2-b7d8-a145a70d7900 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.523692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 298/582] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:32.523692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed54384-afa8-40b1-a66d-1feae307d4d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.612759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcb6ef-c1c9-48e2-b7d8-a145a70d7900 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.629050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed54384-afa8-40b1-a66d-1feae307d4d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.632172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:32.636807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:32.650609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8de4-5888-40a6-b894-cb91edded660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.651227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8de4-5888-40a6-b894-cb91edded660 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.651759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8de4-5888-40a6-b894-cb91edded660 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.652032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da8de4-5888-40a6-b894-cb91edded660 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.653186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da8de4-5888-40a6-b894-cb91edded660 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:32.653782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da8de4-5888-40a6-b894-cb91edded660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:32.655657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da8de4-5888-40a6-b894-cb91edded660 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.663991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 285/583] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:32.667903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e9b3b-8e73-4aa1-b183-0de31b519af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.670052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8de4-5888-40a6-b894-cb91edded660 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.671308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9b3b-8e73-4aa1-b183-0de31b519af1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.671308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9b3b-8e73-4aa1-b183-0de31b519af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.671777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e9b3b-8e73-4aa1-b183-0de31b519af1 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.674225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e9b3b-8e73-4aa1-b183-0de31b519af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:32.675221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e9b3b-8e73-4aa1-b183-0de31b519af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:32.676156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e9b3b-8e73-4aa1-b183-0de31b519af1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.691051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 299/584] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:32.691981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e9b3b-8e73-4aa1-b183-0de31b519af1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.816516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da8de4-5888-40a6-b894-cb91edded660 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.817054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e9b3b-8e73-4aa1-b183-0de31b519af1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:32.854190 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16c786-2fbf-4e7a-874b-f6aa30de0965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.854865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16c786-2fbf-4e7a-874b-f6aa30de0965 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.855308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16c786-2fbf-4e7a-874b-f6aa30de0965 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.855558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16c786-2fbf-4e7a-874b-f6aa30de0965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.862343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82935092-6626-49d8-a243-c5e70e3ed449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:32.863699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82935092-6626-49d8-a243-c5e70e3ed449 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:32.864156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82935092-6626-49d8-a243-c5e70e3ed449 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:32.864602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82935092-6626-49d8-a243-c5e70e3ed449 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:32.865963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82935092-6626-49d8-a243-c5e70e3ed449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:32.866759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82935092-6626-49d8-a243-c5e70e3ed449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:32.868165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82935092-6626-49d8-a243-c5e70e3ed449 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.870099 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16c786-2fbf-4e7a-874b-f6aa30de0965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:32.871559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16c786-2fbf-4e7a-874b-f6aa30de0965 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:32.889134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 286/585] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:32.889134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82935092-6626-49d8-a243-c5e70e3ed449 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-da16c786-2fbf-4e7a-874b-f6aa30de0965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:32.889427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:32.890387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:32.891293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:32.891293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:32.891293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:32.891293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:32.891523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 300/586] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:32.892964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16c786-2fbf-4e7a-874b-f6aa30de0965 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:32.919411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:32.948553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:33.015141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82935092-6626-49d8-a243-c5e70e3ed449 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.023671 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16c786-2fbf-4e7a-874b-f6aa30de0965 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.054632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057a40-5d6b-4f83-8220-e2f68bdfc6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.055281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057a40-5d6b-4f83-8220-e2f68bdfc6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.055584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057a40-5d6b-4f83-8220-e2f68bdfc6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.056168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057a40-5d6b-4f83-8220-e2f68bdfc6f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.068257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62057a40-5d6b-4f83-8220-e2f68bdfc6f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.069630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62057a40-5d6b-4f83-8220-e2f68bdfc6f9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.070352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee07c0-c20f-4ba2-a5f9-13974d5635c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.071291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee07c0-c20f-4ba2-a5f9-13974d5635c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.071961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee07c0-c20f-4ba2-a5f9-13974d5635c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.072816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee07c0-c20f-4ba2-a5f9-13974d5635c0 None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/a1dc3ad467d9496cb3e0086a0cd8215e/roles/4785312981e946918a58301ed5112c34` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-62057a40-5d6b-4f83-8220-e2f68bdfc6f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:33.085829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:33.086781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:33.087718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:33.087718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:33.087718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:33.087718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:33.092591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 287/587] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:33.093451 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057a40-5d6b-4f83-8220-e2f68bdfc6f9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:33.101816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee07c0-c20f-4ba2-a5f9-13974d5635c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=a1dc3ad467d9496cb3e0086a0cd8215e, role_id=4785312981e946918a58301ed5112c34)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.104635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee07c0-c20f-4ba2-a5f9-13974d5635c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.122777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 301/588] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:45:32 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/a1dc3ad467d9496cb3e0086a0cd8215e/roles/4785312981e946918a58301ed5112c34 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:33.123803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee07c0-c20f-4ba2-a5f9-13974d5635c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:33.222029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:33.228744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057a40-5d6b-4f83-8220-e2f68bdfc6f9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.233524 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:33.253944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee07c0-c20f-4ba2-a5f9-13974d5635c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.270142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139121-f1a6-4a07-a1c8-9f52117f8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.270864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139121-f1a6-4a07-a1c8-9f52117f8a63 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.271140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139121-f1a6-4a07-a1c8-9f52117f8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.271411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139121-f1a6-4a07-a1c8-9f52117f8a63 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.282503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97139121-f1a6-4a07-a1c8-9f52117f8a63 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.284132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97139121-f1a6-4a07-a1c8-9f52117f8a63 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-97139121-f1a6-4a07-a1c8-9f52117f8a63 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:33.297732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:33.299005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 288/589] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:33.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139121-f1a6-4a07-a1c8-9f52117f8a63 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:33.302010 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec3020-5afc-429f-b298-962c3fb298d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.302910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec3020-5afc-429f-b298-962c3fb298d6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.303410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec3020-5afc-429f-b298-962c3fb298d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.303889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec3020-5afc-429f-b298-962c3fb298d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.304898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec3020-5afc-429f-b298-962c3fb298d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:33.305517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec3020-5afc-429f-b298-962c3fb298d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.307232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec3020-5afc-429f-b298-962c3fb298d6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.321443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 302/590] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:33.322740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec3020-5afc-429f-b298-962c3fb298d6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:33.424804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139121-f1a6-4a07-a1c8-9f52117f8a63 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.456265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec3020-5afc-429f-b298-962c3fb298d6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.464258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d9ef-cde5-4ca1-a2d2-c2e676e6fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.464983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d9ef-cde5-4ca1-a2d2-c2e676e6fb85 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.465263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d9ef-cde5-4ca1-a2d2-c2e676e6fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.465531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d9ef-cde5-4ca1-a2d2-c2e676e6fb85 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.466223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61d9ef-cde5-4ca1-a2d2-c2e676e6fb85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:33.466773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61d9ef-cde5-4ca1-a2d2-c2e676e6fb85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.468748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61d9ef-cde5-4ca1-a2d2-c2e676e6fb85 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.482661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 289/591] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:33.484118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d9ef-cde5-4ca1-a2d2-c2e676e6fb85 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:33.486665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:33.507774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9981-d6ac-47bb-9c34-456619b3f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.509068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9981-d6ac-47bb-9c34-456619b3f672 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.510020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9981-d6ac-47bb-9c34-456619b3f672 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.510621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9981-d6ac-47bb-9c34-456619b3f672 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.511894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9981-d6ac-47bb-9c34-456619b3f672 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:33.512663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:33.513320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9981-d6ac-47bb-9c34-456619b3f672 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.515440 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9981-d6ac-47bb-9c34-456619b3f672 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.536033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 303/592] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:33.536875 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9981-d6ac-47bb-9c34-456619b3f672 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:33.611841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d9ef-cde5-4ca1-a2d2-c2e676e6fb85 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.658191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d2dee-1c4e-4081-98ec-4516df8cb8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.658955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d2dee-1c4e-4081-98ec-4516df8cb8ac None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.659325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d2dee-1c4e-4081-98ec-4516df8cb8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.659777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d2dee-1c4e-4081-98ec-4516df8cb8ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.666635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9981-d6ac-47bb-9c34-456619b3f672 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.667102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d2dee-1c4e-4081-98ec-4516df8cb8ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.668835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d2dee-1c4e-4081-98ec-4516df8cb8ac None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-241d2dee-1c4e-4081-98ec-4516df8cb8ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:33.681813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:33.682960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:33.684085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:33.684085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:33.684085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:33.684085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:33.684085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:33.684085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:33.684085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:33.684085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:33.685616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 290/593] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:33.686709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d2dee-1c4e-4081-98ec-4516df8cb8ac None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:33.709143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996bd3f-8ddf-4956-97b1-cac40ddf0886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.710091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996bd3f-8ddf-4956-97b1-cac40ddf0886 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.710627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996bd3f-8ddf-4956-97b1-cac40ddf0886 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.711189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996bd3f-8ddf-4956-97b1-cac40ddf0886 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.721186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996bd3f-8ddf-4956-97b1-cac40ddf0886 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.723701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996bd3f-8ddf-4956-97b1-cac40ddf0886 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9996bd3f-8ddf-4956-97b1-cac40ddf0886 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:33.741503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:33.742704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 30 04:45:33.743863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:33.751822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 304/594] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:33.753060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996bd3f-8ddf-4956-97b1-cac40ddf0886 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:33.786561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:33.806344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:33.815541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d2dee-1c4e-4081-98ec-4516df8cb8ac None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.859195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7d39-a551-45bc-a435-ed38fbd0bb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.860178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7d39-a551-45bc-a435-ed38fbd0bb08 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.860460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7d39-a551-45bc-a435-ed38fbd0bb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.860733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7d39-a551-45bc-a435-ed38fbd0bb08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.869805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996bd3f-8ddf-4956-97b1-cac40ddf0886 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:33.874777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be7d39-a551-45bc-a435-ed38fbd0bb08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.876237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be7d39-a551-45bc-a435-ed38fbd0bb08 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-59be7d39-a551-45bc-a435-ed38fbd0bb08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:33.894937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:33.895869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:33.896758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:33.896758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:33.896758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:33.896758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:33.896758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 291/595] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:33.897468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7d39-a551-45bc-a435-ed38fbd0bb08 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:33.911490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9aa34-9369-4762-8d7c-a2357bd08c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:33.912480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9aa34-9369-4762-8d7c-a2357bd08c10 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:33.913064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9aa34-9369-4762-8d7c-a2357bd08c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:33.913530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9aa34-9369-4762-8d7c-a2357bd08c10 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:33.914587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9aa34-9369-4762-8d7c-a2357bd08c10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:33.915091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9aa34-9369-4762-8d7c-a2357bd08c10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:33.916981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9aa34-9369-4762-8d7c-a2357bd08c10 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:33.931814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 305/596] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:33.933048 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9aa34-9369-4762-8d7c-a2357bd08c10 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:34.020497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7d39-a551-45bc-a435-ed38fbd0bb08 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:34.059454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:34.062120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9aa34-9369-4762-8d7c-a2357bd08c10 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:34.065684 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:34.070668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e06d3-3180-4d67-aac7-ae4c19dabe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:34.071294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e06d3-3180-4d67-aac7-ae4c19dabe04 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.071571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e06d3-3180-4d67-aac7-ae4c19dabe04 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.071844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e06d3-3180-4d67-aac7-ae4c19dabe04 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.072640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e06d3-3180-4d67-aac7-ae4c19dabe04 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:34.072944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e06d3-3180-4d67-aac7-ae4c19dabe04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:34.076362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e06d3-3180-4d67-aac7-ae4c19dabe04 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:34.099385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 292/597] 10.222.0.20 () {56 vars in 1139 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:34.100488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e06d3-3180-4d67-aac7-ae4c19dabe04 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:34.115223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94a5e8-ab77-4b07-a83a-e82aeafc4f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:34.115864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94a5e8-ab77-4b07-a83a-e82aeafc4f1e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.116144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94a5e8-ab77-4b07-a83a-e82aeafc4f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.116417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94a5e8-ab77-4b07-a83a-e82aeafc4f1e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.117395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94a5e8-ab77-4b07-a83a-e82aeafc4f1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:34.117705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94a5e8-ab77-4b07-a83a-e82aeafc4f1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:34.119351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94a5e8-ab77-4b07-a83a-e82aeafc4f1e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:34.135135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 306/598] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:34.136286 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94a5e8-ab77-4b07-a83a-e82aeafc4f1e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:34.207639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e06d3-3180-4d67-aac7-ae4c19dabe04 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:34.243687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff50bd-b4d6-49f5-9508-1db089edc87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:34.245129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff50bd-b4d6-49f5-9508-1db089edc87f None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.245580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff50bd-b4d6-49f5-9508-1db089edc87f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.246513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff50bd-b4d6-49f5-9508-1db089edc87f None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/548acf58bf9140ec9f1ba126e10deb45/roles/4785312981e946918a58301ed5112c34` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.263807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff50bd-b4d6-49f5-9508-1db089edc87f None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=548acf58bf9140ec9f1ba126e10deb45, role_id=4785312981e946918a58301ed5112c34)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:34.265470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff50bd-b4d6-49f5-9508-1db089edc87f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:34.271919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94a5e8-ab77-4b07-a83a-e82aeafc4f1e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:34.286346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 293/599] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:45:34 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/548acf58bf9140ec9f1ba126e10deb45/roles/4785312981e946918a58301ed5112c34 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:34.287375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff50bd-b4d6-49f5-9508-1db089edc87f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:34.349593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a960d61-5bff-4a26-9715-1506646801da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:34.350316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a960d61-5bff-4a26-9715-1506646801da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.350628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a960d61-5bff-4a26-9715-1506646801da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.350911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a960d61-5bff-4a26-9715-1506646801da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.371508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:34.379541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a960d61-5bff-4a26-9715-1506646801da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:34.381101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a960d61-5bff-4a26-9715-1506646801da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0a960d61-5bff-4a26-9715-1506646801da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:34.415366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:34.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:34.417173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:34.417173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:34.417173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:34.417173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:34.424032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 307/600] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:34.426335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a960d61-5bff-4a26-9715-1506646801da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:34.448863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:34.480247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff50bd-b4d6-49f5-9508-1db089edc87f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:34.525753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8882d3-8970-4106-89e3-beab87e7921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:34.527047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8882d3-8970-4106-89e3-beab87e7921f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.527398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8882d3-8970-4106-89e3-beab87e7921f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.527738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8882d3-8970-4106-89e3-beab87e7921f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.538186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8882d3-8970-4106-89e3-beab87e7921f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:34.540275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8882d3-8970-4106-89e3-beab87e7921f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:34.553257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a960d61-5bff-4a26-9715-1506646801da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-bd8882d3-8970-4106-89e3-beab87e7921f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:34.556220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:34.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:34.557966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:34.557966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:34.557966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:34.557966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:34.557966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:34.557966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:34.557966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:34.557966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:34.561140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 294/601] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:34.562619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49c1d6-7ea3-4a90-acc7-0d1e1e3c4f3a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.563109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49c1d6-7ea3-4a90-acc7-0d1e1e3c4f3a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.563591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49c1d6-7ea3-4a90-acc7-0d1e1e3c4f3a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.569474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 295/602] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:34.570474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49c1d6-7ea3-4a90-acc7-0d1e1e3c4f3a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:34.604005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412ff2c-7d98-433d-8bac-ff1c461a4cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:34.604716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ff2c-7d98-433d-8bac-ff1c461a4cac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.605057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ff2c-7d98-433d-8bac-ff1c461a4cac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.605331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ff2c-7d98-433d-8bac-ff1c461a4cac None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.606141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ff2c-7d98-433d-8bac-ff1c461a4cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:34.606467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ff2c-7d98-433d-8bac-ff1c461a4cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:34.608402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ff2c-7d98-433d-8bac-ff1c461a4cac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:34.630496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 308/603] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:34.632164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919221-6a4c-4517-8f04-1b477519bb90 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.632455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919221-6a4c-4517-8f04-1b477519bb90 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.632733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7919221-6a4c-4517-8f04-1b477519bb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.659590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a7919221-6a4c-4517-8f04-1b477519bb90 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:34.660831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7919221-6a4c-4517-8f04-1b477519bb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:34.676395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:34.686156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49c1d6-7ea3-4a90-acc7-0d1e1e3c4f3a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:34.727093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb4cdb-e96e-42e4-a58e-4f5ced267acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:34.728140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb4cdb-e96e-42e4-a58e-4f5ced267acd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.728623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb4cdb-e96e-42e4-a58e-4f5ced267acd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.729140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb4cdb-e96e-42e4-a58e-4f5ced267acd None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.730078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb4cdb-e96e-42e4-a58e-4f5ced267acd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:34.730678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb4cdb-e96e-42e4-a58e-4f5ced267acd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:34.733069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb4cdb-e96e-42e4-a58e-4f5ced267acd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:34.746095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 296/604] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:34.747298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb4cdb-e96e-42e4-a58e-4f5ced267acd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:34.755961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 309/605] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:34.756805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7919221-6a4c-4517-8f04-1b477519bb90 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:34.767250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:34.864067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb4cdb-e96e-42e4-a58e-4f5ced267acd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:34.900066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66926a3d-db93-4b64-ac8a-036c038a2aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:34.901055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66926a3d-db93-4b64-ac8a-036c038a2aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.901822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66926a3d-db93-4b64-ac8a-036c038a2aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.901938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7919221-6a4c-4517-8f04-1b477519bb90 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:34.902481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66926a3d-db93-4b64-ac8a-036c038a2aa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.916366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66926a3d-db93-4b64-ac8a-036c038a2aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:34.918143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66926a3d-db93-4b64-ac8a-036c038a2aa0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-66926a3d-db93-4b64-ac8a-036c038a2aa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:34.936782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:34.937667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 297/606] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:34.939182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66926a3d-db93-4b64-ac8a-036c038a2aa0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:34.952424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf60ca-c0e3-406d-9bf6-00bfd26343ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:34.953438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf60ca-c0e3-406d-9bf6-00bfd26343ea None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:34.954571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf60ca-c0e3-406d-9bf6-00bfd26343ea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:34.955082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf60ca-c0e3-406d-9bf6-00bfd26343ea None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/a0d2552bd0e244dea55eb1eecf471fef/roles/4785312981e946918a58301ed5112c34` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:34.964350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:34.978722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf60ca-c0e3-406d-9bf6-00bfd26343ea None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=a0d2552bd0e244dea55eb1eecf471fef, role_id=4785312981e946918a58301ed5112c34)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:34.981428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf60ca-c0e3-406d-9bf6-00bfd26343ea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:35.001181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 310/607] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:45:34 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/a0d2552bd0e244dea55eb1eecf471fef/roles/4785312981e946918a58301ed5112c34 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:35.002626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf60ca-c0e3-406d-9bf6-00bfd26343ea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:35.074178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66926a3d-db93-4b64-ac8a-036c038a2aa0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:35.074847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:35.113474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abb4e6-4c28-4928-8c66-7db433455fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:35.113898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abb4e6-4c28-4928-8c66-7db433455fdb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:35.114186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abb4e6-4c28-4928-8c66-7db433455fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:35.114479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abb4e6-4c28-4928-8c66-7db433455fdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:35.123976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf60ca-c0e3-406d-9bf6-00bfd26343ea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:35.130983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abb4e6-4c28-4928-8c66-7db433455fdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c6abb4e6-4c28-4928-8c66-7db433455fdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:35.148993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:35.150569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:35.151925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 298/608] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:35.159525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abb4e6-4c28-4928-8c66-7db433455fdb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:35.171826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094552c-0c21-4139-8eec-f6c848ffb68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:35.172885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094552c-0c21-4139-8eec-f6c848ffb68c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:35.173450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094552c-0c21-4139-8eec-f6c848ffb68c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:35.173931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094552c-0c21-4139-8eec-f6c848ffb68c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:35.174984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094552c-0c21-4139-8eec-f6c848ffb68c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:35.175507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094552c-0c21-4139-8eec-f6c848ffb68c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:35.177201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094552c-0c21-4139-8eec-f6c848ffb68c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:35.192900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 311/609] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:35.200040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094552c-0c21-4139-8eec-f6c848ffb68c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:35.286051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:35.315004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abb4e6-4c28-4928-8c66-7db433455fdb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:35.353617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063cde6-0c6f-4e5d-9efa-121734a0bbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:35.354230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063cde6-0c6f-4e5d-9efa-121734a0bbdc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:35.354533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063cde6-0c6f-4e5d-9efa-121734a0bbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:35.354802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063cde6-0c6f-4e5d-9efa-121734a0bbdc None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:35.355511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063cde6-0c6f-4e5d-9efa-121734a0bbdc None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:35.355809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063cde6-0c6f-4e5d-9efa-121734a0bbdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:35.357238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063cde6-0c6f-4e5d-9efa-121734a0bbdc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:35.361594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094552c-0c21-4139-8eec-f6c848ffb68c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:35.365949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 299/610] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:35.371204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259affd-f91d-4714-b8fb-fb44ab864b1f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:35.371491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259affd-f91d-4714-b8fb-fb44ab864b1f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:35.371768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259affd-f91d-4714-b8fb-fb44ab864b1f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:35.373932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 300/611] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:35.379191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259affd-f91d-4714-b8fb-fb44ab864b1f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:35.384207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:35.410709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2e3ee-1bb3-43dd-9dc5-082634892f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:35.411236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2e3ee-1bb3-43dd-9dc5-082634892f73 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:35.411514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2e3ee-1bb3-43dd-9dc5-082634892f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:35.411784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2e3ee-1bb3-43dd-9dc5-082634892f73 None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/426be56424504e958402d785a068d2f8/roles/4785312981e946918a58301ed5112c34` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:35.427584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2e3ee-1bb3-43dd-9dc5-082634892f73 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=426be56424504e958402d785a068d2f8, role_id=4785312981e946918a58301ed5112c34)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:35.429031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2e3ee-1bb3-43dd-9dc5-082634892f73 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:35.445461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 312/612] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:45:35 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/426be56424504e958402d785a068d2f8/roles/4785312981e946918a58301ed5112c34 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:35.447388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e84d35-016e-48bd-b826-2819f8fbf5bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:35.447672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e84d35-016e-48bd-b826-2819f8fbf5bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:35.448004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e84d35-016e-48bd-b826-2819f8fbf5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:35.473309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-50e84d35-016e-48bd-b826-2819f8fbf5bb None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:35.474694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50e84d35-016e-48bd-b826-2819f8fbf5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:35.516589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259affd-f91d-4714-b8fb-fb44ab864b1f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:35.593339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714596d-c007-4d75-be92-b4014aba81f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:35.593996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714596d-c007-4d75-be92-b4014aba81f7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:35.594269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714596d-c007-4d75-be92-b4014aba81f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:35.594559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714596d-c007-4d75-be92-b4014aba81f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:35.602216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 313/613] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:35.607912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:35.612343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714596d-c007-4d75-be92-b4014aba81f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:35.614034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714596d-c007-4d75-be92-b4014aba81f7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c714596d-c007-4d75-be92-b4014aba81f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:35.632069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:35.633052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:35.633968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:35.633968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:35.633968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:35.633968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:35.633968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 301/614] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:35.639205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e84d35-016e-48bd-b826-2819f8fbf5bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:35.644383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714596d-c007-4d75-be92-b4014aba81f7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:35.767473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:35.774382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e84d35-016e-48bd-b826-2819f8fbf5bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:35.824936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714596d-c007-4d75-be92-b4014aba81f7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:35.846470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513acd9-0603-4404-abc1-fd8c5a2b0230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:35.848437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513acd9-0603-4404-abc1-fd8c5a2b0230 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:35.849066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513acd9-0603-4404-abc1-fd8c5a2b0230 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:35.850178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513acd9-0603-4404-abc1-fd8c5a2b0230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:35.865919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513acd9-0603-4404-abc1-fd8c5a2b0230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4513acd9-0603-4404-abc1-fd8c5a2b0230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:35.871203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:35.872340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:35.873907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:35.876223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 314/615] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:35.883545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513acd9-0603-4404-abc1-fd8c5a2b0230 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:35.920458 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882dd68-91ca-41d1-8b26-984b3fce7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:35.921119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882dd68-91ca-41d1-8b26-984b3fce7bdf None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:35.921363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882dd68-91ca-41d1-8b26-984b3fce7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:35.921681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882dd68-91ca-41d1-8b26-984b3fce7bdf None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:35.922324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882dd68-91ca-41d1-8b26-984b3fce7bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:35.922646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882dd68-91ca-41d1-8b26-984b3fce7bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:35.924025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882dd68-91ca-41d1-8b26-984b3fce7bdf None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:35.952914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 302/616] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:35.957904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882dd68-91ca-41d1-8b26-984b3fce7bdf None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:36.002107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513acd9-0603-4404-abc1-fd8c5a2b0230 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:36.041946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c63232-8047-4a5d-9e85-e513b2220e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:36.042900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c63232-8047-4a5d-9e85-e513b2220e48 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:36.043363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c63232-8047-4a5d-9e85-e513b2220e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:36.043833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c63232-8047-4a5d-9e85-e513b2220e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:36.044895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c63232-8047-4a5d-9e85-e513b2220e48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:36.045381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c63232-8047-4a5d-9e85-e513b2220e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:36.047066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c63232-8047-4a5d-9e85-e513b2220e48 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:36.056418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 315/617] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:36.065173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c63232-8047-4a5d-9e85-e513b2220e48 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:36.072685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:36.102881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:36.133559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882dd68-91ca-41d1-8b26-984b3fce7bdf None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:36.175479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d259f8-eeb8-4638-ace1-3e14d8dce337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:36.176440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d259f8-eeb8-4638-ace1-3e14d8dce337 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:36.176936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d259f8-eeb8-4638-ace1-3e14d8dce337 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:36.177382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d259f8-eeb8-4638-ace1-3e14d8dce337 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:36.186924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d259f8-eeb8-4638-ace1-3e14d8dce337 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:36.189262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d259f8-eeb8-4638-ace1-3e14d8dce337 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-59d259f8-eeb8-4638-ace1-3e14d8dce337 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:36.204530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:36.205505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:36.206469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:36.208969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 303/618] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:36.213856 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d259f8-eeb8-4638-ace1-3e14d8dce337 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:36.270261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c63232-8047-4a5d-9e85-e513b2220e48 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:36.337762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f0c12-821a-4266-9386-206bd1695bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:36.338688 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f0c12-821a-4266-9386-206bd1695bad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:36.339279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f0c12-821a-4266-9386-206bd1695bad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:36.340732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f0c12-821a-4266-9386-206bd1695bad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:36.354217 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d259f8-eeb8-4638-ace1-3e14d8dce337 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:36.365204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f0c12-821a-4266-9386-206bd1695bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:36.367100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f0c12-821a-4266-9386-206bd1695bad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:36.382857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-581f0c12-821a-4266-9386-206bd1695bad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:36.392169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:36.393302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:36.394485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:36.394485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:36.394485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:36.394485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:36.396450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 316/619] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:36.402280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f0c12-821a-4266-9386-206bd1695bad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:36.448116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9c838-e06c-4c4c-ae7e-1266989cea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:36.448947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9c838-e06c-4c4c-ae7e-1266989cea98 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:36.449070 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9c838-e06c-4c4c-ae7e-1266989cea98 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:36.449291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9c838-e06c-4c4c-ae7e-1266989cea98 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:36.450094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9c838-e06c-4c4c-ae7e-1266989cea98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:36.450398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9c838-e06c-4c4c-ae7e-1266989cea98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:36.452469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9c838-e06c-4c4c-ae7e-1266989cea98 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:36.480583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:36.490521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 304/620] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Jul 30 04:45:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:36.493238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9c838-e06c-4c4c-ae7e-1266989cea98 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:36.519925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f0c12-821a-4266-9386-206bd1695bad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:36.555605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada7521-ad23-427a-b00b-877b2f69d06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:36.556461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada7521-ad23-427a-b00b-877b2f69d06f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:36.556946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada7521-ad23-427a-b00b-877b2f69d06f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:36.557357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada7521-ad23-427a-b00b-877b2f69d06f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:36.558328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada7521-ad23-427a-b00b-877b2f69d06f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:36.558789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada7521-ad23-427a-b00b-877b2f69d06f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:36.561004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada7521-ad23-427a-b00b-877b2f69d06f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:36.573741 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 317/621] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:36.579866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada7521-ad23-427a-b00b-877b2f69d06f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:36.682682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9c838-e06c-4c4c-ae7e-1266989cea98 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:36.724382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467c50f-02b5-408c-a57e-450a667d5513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:36.725282 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467c50f-02b5-408c-a57e-450a667d5513 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:36.725858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467c50f-02b5-408c-a57e-450a667d5513 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:36.726467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467c50f-02b5-408c-a57e-450a667d5513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:36.738675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467c50f-02b5-408c-a57e-450a667d5513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:36.740858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467c50f-02b5-408c-a57e-450a667d5513 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4467c50f-02b5-408c-a57e-450a667d5513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:36.758075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:36.759249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:36.760412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:36.760412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:36.760412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:36.760412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:36.762391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 305/622] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:36.768313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467c50f-02b5-408c-a57e-450a667d5513 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:36.778999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada7521-ad23-427a-b00b-877b2f69d06f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:36.814914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:36.817877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:36.824198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5162-f2f1-4713-9311-676aed3b505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:36.827492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5162-f2f1-4713-9311-676aed3b505f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:36.828730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5162-f2f1-4713-9311-676aed3b505f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:36.830176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf5162-f2f1-4713-9311-676aed3b505f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:36.846352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf5162-f2f1-4713-9311-676aed3b505f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:36.849247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf5162-f2f1-4713-9311-676aed3b505f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9ebf5162-f2f1-4713-9311-676aed3b505f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:36.868131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:36.869347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:36.870559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:36.873003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 318/623] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Jul 30 04:45:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:36.879186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5162-f2f1-4713-9311-676aed3b505f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:36.953377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467c50f-02b5-408c-a57e-450a667d5513 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:36.996998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a398fb-ff26-4408-b48b-68fae3c028c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:36.998230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a398fb-ff26-4408-b48b-68fae3c028c8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:36.998704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a398fb-ff26-4408-b48b-68fae3c028c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:36.998975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a398fb-ff26-4408-b48b-68fae3c028c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:36.999670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a398fb-ff26-4408-b48b-68fae3c028c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:37.000302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a398fb-ff26-4408-b48b-68fae3c028c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:37.001941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a398fb-ff26-4408-b48b-68fae3c028c8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:37.009265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf5162-f2f1-4713-9311-676aed3b505f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:37.018478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 306/624] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:37.020345 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dde675-79b0-4cf4-80ca-2a97b004610e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.020655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dde675-79b0-4cf4-80ca-2a97b004610e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.020938 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dde675-79b0-4cf4-80ca-2a97b004610e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.026917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 307/625] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:37.028225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dde675-79b0-4cf4-80ca-2a97b004610e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:37.054959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a618c8-a8e7-4726-81c6-422c71f41b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:37.055962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a618c8-a8e7-4726-81c6-422c71f41b3c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.056396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a618c8-a8e7-4726-81c6-422c71f41b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.056848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a618c8-a8e7-4726-81c6-422c71f41b3c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.057816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a618c8-a8e7-4726-81c6-422c71f41b3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:37.058354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a618c8-a8e7-4726-81c6-422c71f41b3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:37.060184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a618c8-a8e7-4726-81c6-422c71f41b3c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:37.085401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 319/626] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Jul 30 04:45:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:37.087031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc355c-b768-42b4-acd5-571dc684842a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.087499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc355c-b768-42b4-acd5-571dc684842a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.087932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc355c-b768-42b4-acd5-571dc684842a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.108516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6ffc355c-b768-42b4-acd5-571dc684842a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:37.109941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ffc355c-b768-42b4-acd5-571dc684842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:37.125285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:37.151413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dde675-79b0-4cf4-80ca-2a97b004610e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:37.166812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:37.195173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378186a-4c9a-405f-a2dc-bfff480abe5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:37.196108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378186a-4c9a-405f-a2dc-bfff480abe5c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.196539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378186a-4c9a-405f-a2dc-bfff480abe5c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.196987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378186a-4c9a-405f-a2dc-bfff480abe5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.197367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 320/627] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:37.203476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc355c-b768-42b4-acd5-571dc684842a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:37.213512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378186a-4c9a-405f-a2dc-bfff480abe5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:37.219076 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378186a-4c9a-405f-a2dc-bfff480abe5c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e378186a-4c9a-405f-a2dc-bfff480abe5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:37.236893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:37.238054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:37.239246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:37.239246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:37.239246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:37.239246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:37.239246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:37.239246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:37.239246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:37.239246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:37.242085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 308/628] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:37.243373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378186a-4c9a-405f-a2dc-bfff480abe5c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:37.338177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc355c-b768-42b4-acd5-571dc684842a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:37.369355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378186a-4c9a-405f-a2dc-bfff480abe5c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:37.379016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476152a-56bd-43cf-8d4e-8430d9caa0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:37.379788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476152a-56bd-43cf-8d4e-8430d9caa0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.380413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476152a-56bd-43cf-8d4e-8430d9caa0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.380761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476152a-56bd-43cf-8d4e-8430d9caa0c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.407382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ff740-5cab-4b84-85d6-89db13db9cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:37.408025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ff740-5cab-4b84-85d6-89db13db9cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.408306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ff740-5cab-4b84-85d6-89db13db9cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.408576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ff740-5cab-4b84-85d6-89db13db9cb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.416854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:37.421184 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ff740-5cab-4b84-85d6-89db13db9cb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-126ff740-5cab-4b84-85d6-89db13db9cb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:37.427198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:37.428368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:37.429260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 309/629] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:37.429984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ff740-5cab-4b84-85d6-89db13db9cb3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:37.438611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:37.463356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476152a-56bd-43cf-8d4e-8430d9caa0c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:37.464733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476152a-56bd-43cf-8d4e-8430d9caa0c2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:37.537191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ff740-5cab-4b84-85d6-89db13db9cb3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:37.569180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 321/630] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:37.571990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d5188-735c-41a2-980c-202913af039c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.571990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d5188-735c-41a2-980c-202913af039c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.571990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d5188-735c-41a2-980c-202913af039c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.576409 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 322/631] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:37.577476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d5188-735c-41a2-980c-202913af039c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:37.592206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162ca77-3a70-41ce-aed9-7b488ce511f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:37.593122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162ca77-3a70-41ce-aed9-7b488ce511f1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.593584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162ca77-3a70-41ce-aed9-7b488ce511f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.594051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162ca77-3a70-41ce-aed9-7b488ce511f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.595024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162ca77-3a70-41ce-aed9-7b488ce511f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:37.595524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162ca77-3a70-41ce-aed9-7b488ce511f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:37.597140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162ca77-3a70-41ce-aed9-7b488ce511f1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:37.610609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 310/632] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:37.611493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162ca77-3a70-41ce-aed9-7b488ce511f1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:37.666071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:37.674912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:37.696751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d5188-735c-41a2-980c-202913af039c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:37.722093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162ca77-3a70-41ce-aed9-7b488ce511f1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:37.735620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a859a-d864-44b1-965e-5a1f4df858eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:37.736238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a859a-d864-44b1-965e-5a1f4df858eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.736526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a859a-d864-44b1-965e-5a1f4df858eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.736799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a859a-d864-44b1-965e-5a1f4df858eb None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.737514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a859a-d864-44b1-965e-5a1f4df858eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:37.737814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a859a-d864-44b1-965e-5a1f4df858eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:37.739239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a859a-d864-44b1-965e-5a1f4df858eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:37.748596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 323/633] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:37.749911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e653b-621b-49cd-949d-e07c0e4bb816 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.750199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e653b-621b-49cd-949d-e07c0e4bb816 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.750496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e653b-621b-49cd-949d-e07c0e4bb816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.765870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93960d9-c157-433e-87cb-6b1ca0c2e751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:37.766780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93960d9-c157-433e-87cb-6b1ca0c2e751 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.767228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93960d9-c157-433e-87cb-6b1ca0c2e751 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.767648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93960d9-c157-433e-87cb-6b1ca0c2e751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.768975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5f8e653b-621b-49cd-949d-e07c0e4bb816 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:37.770163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f8e653b-621b-49cd-949d-e07c0e4bb816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:37.779722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93960d9-c157-433e-87cb-6b1ca0c2e751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:37.781338 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93960d9-c157-433e-87cb-6b1ca0c2e751 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a93960d9-c157-433e-87cb-6b1ca0c2e751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:37.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:37.798154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:37.799258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:37.799258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:37.799258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:37.799258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:37.800746 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 311/634] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:37.801668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93960d9-c157-433e-87cb-6b1ca0c2e751 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:37.848528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 324/635] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:37.849380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e653b-621b-49cd-949d-e07c0e4bb816 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:37.916254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93960d9-c157-433e-87cb-6b1ca0c2e751 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:37.919518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:37.928899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:37.951288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e653b-621b-49cd-949d-e07c0e4bb816 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:37.951434 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf62bc-90f5-43db-99b8-6e85397ffba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:37.952093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf62bc-90f5-43db-99b8-6e85397ffba7 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.952362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf62bc-90f5-43db-99b8-6e85397ffba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.952630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf62bc-90f5-43db-99b8-6e85397ffba7 None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/54bda2af62ee4ee39a9a350b7ce6430e/roles/4785312981e946918a58301ed5112c34` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:37.967165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf62bc-90f5-43db-99b8-6e85397ffba7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=54bda2af62ee4ee39a9a350b7ce6430e, role_id=4785312981e946918a58301ed5112c34)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:37.968589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf62bc-90f5-43db-99b8-6e85397ffba7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:37.990600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 312/636] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:45:37 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/54bda2af62ee4ee39a9a350b7ce6430e/roles/4785312981e946918a58301ed5112c34 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:37.990600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf62bc-90f5-43db-99b8-6e85397ffba7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:37.996362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c71769-7cf0-424d-a2e5-c449bf9792f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:37.997042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c71769-7cf0-424d-a2e5-c449bf9792f5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:37.997327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c71769-7cf0-424d-a2e5-c449bf9792f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:37.998210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c71769-7cf0-424d-a2e5-c449bf9792f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.012042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c71769-7cf0-424d-a2e5-c449bf9792f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:38.014084 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c71769-7cf0-424d-a2e5-c449bf9792f5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-33c71769-7cf0-424d-a2e5-c449bf9792f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:38.033582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:38.034885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:38.035947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:38.035947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:38.035947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:38.035947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:38.039858 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 325/637] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:38.041337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983fe4b-9e78-42cc-ab9f-5162934c1b48 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.041628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983fe4b-9e78-42cc-ab9f-5162934c1b48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.041896 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983fe4b-9e78-42cc-ab9f-5162934c1b48 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.043543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 326/638] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:38.043983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983fe4b-9e78-42cc-ab9f-5162934c1b48 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.108838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf62bc-90f5-43db-99b8-6e85397ffba7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.152375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a50bce-6f3a-4f02-9afb-59d6bacf9f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:38.152975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a50bce-6f3a-4f02-9afb-59d6bacf9f89 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.153252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a50bce-6f3a-4f02-9afb-59d6bacf9f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.153528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a50bce-6f3a-4f02-9afb-59d6bacf9f89 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.154813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a50bce-6f3a-4f02-9afb-59d6bacf9f89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:38.155114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a50bce-6f3a-4f02-9afb-59d6bacf9f89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:38.156502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a50bce-6f3a-4f02-9afb-59d6bacf9f89 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:38.171116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 313/639] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:38.171116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a50bce-6f3a-4f02-9afb-59d6bacf9f89 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.175454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983fe4b-9e78-42cc-ab9f-5162934c1b48 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.195780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:38.199807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:38.213041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87dd59-5a98-4ac4-b443-9a3c12f79fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:38.213694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87dd59-5a98-4ac4-b443-9a3c12f79fcc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.213974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87dd59-5a98-4ac4-b443-9a3c12f79fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.214251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87dd59-5a98-4ac4-b443-9a3c12f79fcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.226079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87dd59-5a98-4ac4-b443-9a3c12f79fcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0a87dd59-5a98-4ac4-b443-9a3c12f79fcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:38.231578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:38.233208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:38.234054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 327/640] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:38.234817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8357d-fb39-4f12-93bb-f171ad21ebae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.234886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8357d-fb39-4f12-93bb-f171ad21ebae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.235608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8357d-fb39-4f12-93bb-f171ad21ebae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.253770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dfe8357d-fb39-4f12-93bb-f171ad21ebae None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:38.254940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dfe8357d-fb39-4f12-93bb-f171ad21ebae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:38.287048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a50bce-6f3a-4f02-9afb-59d6bacf9f89 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.322981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4697d5c-e1cf-49b6-a88b-9327dbc760e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:38.323610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4697d5c-e1cf-49b6-a88b-9327dbc760e6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.323889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4697d5c-e1cf-49b6-a88b-9327dbc760e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.324158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4697d5c-e1cf-49b6-a88b-9327dbc760e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.324846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4697d5c-e1cf-49b6-a88b-9327dbc760e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:38.325149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4697d5c-e1cf-49b6-a88b-9327dbc760e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:38.330491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 328/641] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:38.330636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4697d5c-e1cf-49b6-a88b-9327dbc760e6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:38.330735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8357d-fb39-4f12-93bb-f171ad21ebae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.346632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 314/642] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:38.349108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4697d5c-e1cf-49b6-a88b-9327dbc760e6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.440561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:38.444073 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8357d-fb39-4f12-93bb-f171ad21ebae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.460203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:38.463313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4697d5c-e1cf-49b6-a88b-9327dbc760e6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.487783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60658cd-a918-41e9-82c1-9fe92be9cc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:38.488457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60658cd-a918-41e9-82c1-9fe92be9cc39 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.488746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60658cd-a918-41e9-82c1-9fe92be9cc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.489019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60658cd-a918-41e9-82c1-9fe92be9cc39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.496326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60658cd-a918-41e9-82c1-9fe92be9cc39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:38.498892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60658cd-a918-41e9-82c1-9fe92be9cc39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:38.500183 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278eecb-23b8-4e43-bb95-0419b1edfffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:38.500805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278eecb-23b8-4e43-bb95-0419b1edfffa None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.501083 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278eecb-23b8-4e43-bb95-0419b1edfffa None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.501360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278eecb-23b8-4e43-bb95-0419b1edfffa None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.502074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278eecb-23b8-4e43-bb95-0419b1edfffa None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:38.502373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278eecb-23b8-4e43-bb95-0419b1edfffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:38.503824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278eecb-23b8-4e43-bb95-0419b1edfffa None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a60658cd-a918-41e9-82c1-9fe92be9cc39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:38.511416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:38.512348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:38.513880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:38.513880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:38.513880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:38.513880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:38.513880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:38.513880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:38.513880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:38.513880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:38.513880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 329/643] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:38.514384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60658cd-a918-41e9-82c1-9fe92be9cc39 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.516564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 315/644] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:38.517310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278eecb-23b8-4e43-bb95-0419b1edfffa None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.624915 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60658cd-a918-41e9-82c1-9fe92be9cc39 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.637699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278eecb-23b8-4e43-bb95-0419b1edfffa None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.662513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876a5d3-9888-4fc7-a70a-39adc46da9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:38.663162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876a5d3-9888-4fc7-a70a-39adc46da9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.663445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876a5d3-9888-4fc7-a70a-39adc46da9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.663716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876a5d3-9888-4fc7-a70a-39adc46da9c9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.671442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876a5d3-9888-4fc7-a70a-39adc46da9c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:38.672899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876a5d3-9888-4fc7-a70a-39adc46da9c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:38.677113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c55e6e-cf49-4245-af7d-8eb9b6f639a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:38.677871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c55e6e-cf49-4245-af7d-8eb9b6f639a9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.678241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c55e6e-cf49-4245-af7d-8eb9b6f639a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.678634 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c55e6e-cf49-4245-af7d-8eb9b6f639a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5876a5d3-9888-4fc7-a70a-39adc46da9c9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:38.685588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:38.686758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 30 04:45:38.688104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:38.691652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 330/645] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:38.692632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876a5d3-9888-4fc7-a70a-39adc46da9c9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.693844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c55e6e-cf49-4245-af7d-8eb9b6f639a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-37c55e6e-cf49-4245-af7d-8eb9b6f639a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:38.699356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:38.701026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:38.702104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:38.703876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 316/646] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:38.703876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c55e6e-cf49-4245-af7d-8eb9b6f639a9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.708568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:38.731119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:38.801144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876a5d3-9888-4fc7-a70a-39adc46da9c9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.813535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c55e6e-cf49-4245-af7d-8eb9b6f639a9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.836149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f957b5-4acd-4aeb-ad9d-cf1225481e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:38.836788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f957b5-4acd-4aeb-ad9d-cf1225481e8b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.837070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f957b5-4acd-4aeb-ad9d-cf1225481e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.837350 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f957b5-4acd-4aeb-ad9d-cf1225481e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.848103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df26c29-8219-4956-bac3-8de96b32da14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:38.848300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f957b5-4acd-4aeb-ad9d-cf1225481e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:38.848834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df26c29-8219-4956-bac3-8de96b32da14 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:38.849172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df26c29-8219-4956-bac3-8de96b32da14 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:38.849499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df26c29-8219-4956-bac3-8de96b32da14 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:38.849715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f957b5-4acd-4aeb-ad9d-cf1225481e8b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:38.850404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df26c29-8219-4956-bac3-8de96b32da14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:38.850743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df26c29-8219-4956-bac3-8de96b32da14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:38.852406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df26c29-8219-4956-bac3-8de96b32da14 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a8f957b5-4acd-4aeb-ad9d-cf1225481e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:38.864288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:38.865305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:38.866396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:38.866396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:38.866396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:38.866396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:38.866692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 317/647] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:38.867081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df26c29-8219-4956-bac3-8de96b32da14 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.869800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 331/648] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:38.870671 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f957b5-4acd-4aeb-ad9d-cf1225481e8b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:38.934521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:38.967391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:38.982449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df26c29-8219-4956-bac3-8de96b32da14 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:38.985521 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f957b5-4acd-4aeb-ad9d-cf1225481e8b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:39.024647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c8b35-6763-488b-8e6e-01a0ecdeec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.025369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c8b35-6763-488b-8e6e-01a0ecdeec1e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.025622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb0ec9-c002-4d9d-be9e-ddddc6f8e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.025756 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c8b35-6763-488b-8e6e-01a0ecdeec1e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.025908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c8b35-6763-488b-8e6e-01a0ecdeec1e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.026678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb0ec9-c002-4d9d-be9e-ddddc6f8e88e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.026964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c8b35-6763-488b-8e6e-01a0ecdeec1e None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:39.027097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb0ec9-c002-4d9d-be9e-ddddc6f8e88e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.027263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c8b35-6763-488b-8e6e-01a0ecdeec1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.027475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb0ec9-c002-4d9d-be9e-ddddc6f8e88e None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.028327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb0ec9-c002-4d9d-be9e-ddddc6f8e88e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:39.029168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c8b35-6763-488b-8e6e-01a0ecdeec1e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.029607 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb0ec9-c002-4d9d-be9e-ddddc6f8e88e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.031152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb0ec9-c002-4d9d-be9e-ddddc6f8e88e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.043789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 332/649] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:39.043789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c8b35-6763-488b-8e6e-01a0ecdeec1e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.050924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 318/650] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Jul 30 04:45:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:39.054556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb0ec9-c002-4d9d-be9e-ddddc6f8e88e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.156583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c8b35-6763-488b-8e6e-01a0ecdeec1e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:39.157876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb0ec9-c002-4d9d-be9e-ddddc6f8e88e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:39.181377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:39.195274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e9a2f-1263-44e0-acf6-0b83e1497833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.195470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837aa35c-ca36-4ba0-ba7d-eba77c1bf2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.196050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e9a2f-1263-44e0-acf6-0b83e1497833 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.196200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aa35c-ca36-4ba0-ba7d-eba77c1bf2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.196450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e9a2f-1263-44e0-acf6-0b83e1497833 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.196542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aa35c-ca36-4ba0-ba7d-eba77c1bf2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.196755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aa35c-ca36-4ba0-ba7d-eba77c1bf2e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.196857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e9a2f-1263-44e0-acf6-0b83e1497833 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.197575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837aa35c-ca36-4ba0-ba7d-eba77c1bf2e1 None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=a1dc3ad467d9496cb3e0086a0cd8215e) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:39.197671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e9a2f-1263-44e0-acf6-0b83e1497833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:39.197884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837aa35c-ca36-4ba0-ba7d-eba77c1bf2e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.198081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e9a2f-1263-44e0-acf6-0b83e1497833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.199355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837aa35c-ca36-4ba0-ba7d-eba77c1bf2e1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.199603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e9a2f-1263-44e0-acf6-0b83e1497833 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.213304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 333/651] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/role_assignments?user.id=a1dc3ad467d9496cb3e0086a0cd8215e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:39.213304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837aa35c-ca36-4ba0-ba7d-eba77c1bf2e1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.213598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 319/652] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:39.213598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e9a2f-1263-44e0-acf6-0b83e1497833 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.236028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:39.334173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e9a2f-1263-44e0-acf6-0b83e1497833 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:39.363149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837aa35c-ca36-4ba0-ba7d-eba77c1bf2e1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:39.378552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9426918-4f49-416e-84e9-a1ad4094b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.379393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9426918-4f49-416e-84e9-a1ad4094b56f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.379725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9426918-4f49-416e-84e9-a1ad4094b56f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.380070 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9426918-4f49-416e-84e9-a1ad4094b56f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.408707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919f4d7-1ebb-410f-9518-bbed1c4f684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.409518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919f4d7-1ebb-410f-9518-bbed1c4f684a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.409802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919f4d7-1ebb-410f-9518-bbed1c4f684a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.410083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919f4d7-1ebb-410f-9518-bbed1c4f684a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.422859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919f4d7-1ebb-410f-9518-bbed1c4f684a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.425280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919f4d7-1ebb-410f-9518-bbed1c4f684a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.442191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9919f4d7-1ebb-410f-9518-bbed1c4f684a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:39.442421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:39.443345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:39.444625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:39.444625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:39.444625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:39.444625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:39.444625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 334/653] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:39.444910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919f4d7-1ebb-410f-9518-bbed1c4f684a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.469383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9426918-4f49-416e-84e9-a1ad4094b56f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.470958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9426918-4f49-416e-84e9-a1ad4094b56f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.537775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:39.577243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919f4d7-1ebb-410f-9518-bbed1c4f684a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:39.600017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 320/654] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:39.600950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9426918-4f49-416e-84e9-a1ad4094b56f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.616811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061e3e0-1216-4855-a594-43a3c6c0e1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.617819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061e3e0-1216-4855-a594-43a3c6c0e1fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.618306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061e3e0-1216-4855-a594-43a3c6c0e1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.618856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061e3e0-1216-4855-a594-43a3c6c0e1fb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.628103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061e3e0-1216-4855-a594-43a3c6c0e1fb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.629863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061e3e0-1216-4855-a594-43a3c6c0e1fb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f061e3e0-1216-4855-a594-43a3c6c0e1fb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:39.643637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:39.644611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 30 04:45:39.645575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:39.648201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 335/655] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:39.649274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061e3e0-1216-4855-a594-43a3c6c0e1fb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.702811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:39.727261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9426918-4f49-416e-84e9-a1ad4094b56f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:39.764184 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b01dd-8db5-45c2-aebf-bc123157ca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.764983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b01dd-8db5-45c2-aebf-bc123157ca55 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.765261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b01dd-8db5-45c2-aebf-bc123157ca55 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.765531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b01dd-8db5-45c2-aebf-bc123157ca55 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.766173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061e3e0-1216-4855-a594-43a3c6c0e1fb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:39.766285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b01dd-8db5-45c2-aebf-bc123157ca55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:39.766556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b01dd-8db5-45c2-aebf-bc123157ca55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.767918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b01dd-8db5-45c2-aebf-bc123157ca55 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.780377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 321/656] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:39.781299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b01dd-8db5-45c2-aebf-bc123157ca55 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.794737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:39.809609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd2fef-ce68-4583-83f4-824d4102e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.811500 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd2fef-ce68-4583-83f4-824d4102e127 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.812049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd2fef-ce68-4583-83f4-824d4102e127 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.812525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd2fef-ce68-4583-83f4-824d4102e127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.824464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd2fef-ce68-4583-83f4-824d4102e127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.827063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd2fef-ce68-4583-83f4-824d4102e127 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c0bd2fef-ce68-4583-83f4-824d4102e127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:39.844733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:39.845685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:39.846605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:39.846605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:39.846605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:39.846605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:39.848799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 336/657] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:39.849787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd2fef-ce68-4583-83f4-824d4102e127 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.885196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b01dd-8db5-45c2-aebf-bc123157ca55 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:39.924768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ba1ab-33f5-415a-b83b-dbabba8b44de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:39.925763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ba1ab-33f5-415a-b83b-dbabba8b44de None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:39.926260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ba1ab-33f5-415a-b83b-dbabba8b44de None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:39.926755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ba1ab-33f5-415a-b83b-dbabba8b44de None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:39.927751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ba1ab-33f5-415a-b83b-dbabba8b44de None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:39.928255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ba1ab-33f5-415a-b83b-dbabba8b44de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:39.929920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ba1ab-33f5-415a-b83b-dbabba8b44de None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:39.946707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:39.952729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 322/658] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:39.954981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ba1ab-33f5-415a-b83b-dbabba8b44de None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:39.979569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd2fef-ce68-4583-83f4-824d4102e127 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.022223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc746de-95f7-4d94-bc68-aec3df2db7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:40.025387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc746de-95f7-4d94-bc68-aec3df2db7df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.025387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc746de-95f7-4d94-bc68-aec3df2db7df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.025387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc746de-95f7-4d94-bc68-aec3df2db7df None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.035744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc746de-95f7-4d94-bc68-aec3df2db7df None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:40.037412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc746de-95f7-4d94-bc68-aec3df2db7df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7fc746de-95f7-4d94-bc68-aec3df2db7df None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:40.049813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:40.050973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 30 04:45:40.052126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:40.057836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 337/659] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:40.057836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc746de-95f7-4d94-bc68-aec3df2db7df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:40.090904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:40.093293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ba1ab-33f5-415a-b83b-dbabba8b44de None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.131666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292e227-10da-47ec-aedf-c4ed24ac4afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:40.133047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292e227-10da-47ec-aedf-c4ed24ac4afd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.133321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292e227-10da-47ec-aedf-c4ed24ac4afd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.133587 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292e227-10da-47ec-aedf-c4ed24ac4afd None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.134270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292e227-10da-47ec-aedf-c4ed24ac4afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:40.134773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292e227-10da-47ec-aedf-c4ed24ac4afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:40.136197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292e227-10da-47ec-aedf-c4ed24ac4afd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:40.148919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 323/660] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:40.150163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f58d3-aa25-4531-ae4e-d383e8e8817b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.150473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f58d3-aa25-4531-ae4e-d383e8e8817b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.150739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f58d3-aa25-4531-ae4e-d383e8e8817b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.154467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 324/661] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:40.154467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f58d3-aa25-4531-ae4e-d383e8e8817b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:40.175522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc746de-95f7-4d94-bc68-aec3df2db7df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.212283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecb8cb-b086-496e-8033-db2dcea2f3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:40.212974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecb8cb-b086-496e-8033-db2dcea2f3be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.213255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecb8cb-b086-496e-8033-db2dcea2f3be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.213526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecb8cb-b086-496e-8033-db2dcea2f3be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.219360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:40.266507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f58d3-aa25-4531-ae4e-d383e8e8817b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.284443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecb8cb-b086-496e-8033-db2dcea2f3be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:40.285898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecb8cb-b086-496e-8033-db2dcea2f3be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:40.305724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1072a-4f69-432b-9993-0bcf1bacadc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:40.306508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1072a-4f69-432b-9993-0bcf1bacadc7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.306892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1072a-4f69-432b-9993-0bcf1bacadc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.307266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1072a-4f69-432b-9993-0bcf1bacadc7 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.308157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1072a-4f69-432b-9993-0bcf1bacadc7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:40.308563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1072a-4f69-432b-9993-0bcf1bacadc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:40.310771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1072a-4f69-432b-9993-0bcf1bacadc7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:40.321484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:40.330729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 325/662] 10.222.0.20 () {56 vars in 1139 bytes} [Sat Jul 30 04:45:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:40.331620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1072a-4f69-432b-9993-0bcf1bacadc7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:40.402565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 338/663] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:40.402565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf968c-487f-4cf9-b724-61fd9aa46662 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.402565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf968c-487f-4cf9-b724-61fd9aa46662 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.403205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf968c-487f-4cf9-b724-61fd9aa46662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.422772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2caf968c-487f-4cf9-b724-61fd9aa46662 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:40.424103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2caf968c-487f-4cf9-b724-61fd9aa46662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:40.437027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1072a-4f69-432b-9993-0bcf1bacadc7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.456395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:40.475202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d95c5-fead-4de9-8202-11089e4059f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:40.475565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d95c5-fead-4de9-8202-11089e4059f6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.475892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d95c5-fead-4de9-8202-11089e4059f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.476154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d95c5-fead-4de9-8202-11089e4059f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.483311 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d95c5-fead-4de9-8202-11089e4059f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:40.484733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d95c5-fead-4de9-8202-11089e4059f6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:40.506493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 339/664] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-803d95c5-fead-4de9-8202-11089e4059f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:40.506705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:40.507700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:40.508804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:40.508804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:40.508804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:40.508804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:40.508804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:40.508804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:40.508804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:40.508804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:40.509332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf968c-487f-4cf9-b724-61fd9aa46662 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:40.509415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 326/665] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:40.509415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d95c5-fead-4de9-8202-11089e4059f6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:40.584104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:40.617611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d95c5-fead-4de9-8202-11089e4059f6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.624798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf968c-487f-4cf9-b724-61fd9aa46662 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.652292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7879-cd69-4aff-9f58-5aec02b4ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:40.652977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7879-cd69-4aff-9f58-5aec02b4ee2a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.653199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7879-cd69-4aff-9f58-5aec02b4ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.653509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7879-cd69-4aff-9f58-5aec02b4ee2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.663348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a7879-cd69-4aff-9f58-5aec02b4ee2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:40.664779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a7879-cd69-4aff-9f58-5aec02b4ee2a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:40.665066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff243a-f89b-4bd6-bcd7-e55c0d35d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:40.665993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff243a-f89b-4bd6-bcd7-e55c0d35d2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.666583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff243a-f89b-4bd6-bcd7-e55c0d35d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.667021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff243a-f89b-4bd6-bcd7-e55c0d35d2a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a24a7879-cd69-4aff-9f58-5aec02b4ee2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:40.679830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:40.680926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:40.682064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:40.682064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:40.682064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:40.682064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:40.685859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 327/666] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:40.687135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7879-cd69-4aff-9f58-5aec02b4ee2a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:40.716665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:40.745543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff243a-f89b-4bd6-bcd7-e55c0d35d2a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:40.747044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff243a-f89b-4bd6-bcd7-e55c0d35d2a2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:40.786934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7879-cd69-4aff-9f58-5aec02b4ee2a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.813223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:40.826778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016315-01cd-41d2-a573-9418cc35b8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:40.827392 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016315-01cd-41d2-a573-9418cc35b8ea None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.827665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016315-01cd-41d2-a573-9418cc35b8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:40.827933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f016315-01cd-41d2-a573-9418cc35b8ea None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:40.828678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f016315-01cd-41d2-a573-9418cc35b8ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:40.828977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f016315-01cd-41d2-a573-9418cc35b8ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:40.830407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f016315-01cd-41d2-a573-9418cc35b8ea None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:40.848202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 328/667] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:40.848202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016315-01cd-41d2-a573-9418cc35b8ea None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:40.867117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 340/668] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:40.867117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff243a-f89b-4bd6-bcd7-e55c0d35d2a2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:40.942701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:40.955008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f016315-01cd-41d2-a573-9418cc35b8ea None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.970967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff243a-f89b-4bd6-bcd7-e55c0d35d2a2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:40.998216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2150-d9a2-4468-a8e9-e48b21650109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:40.999693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2150-d9a2-4468-a8e9-e48b21650109 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:40.999975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2150-d9a2-4468-a8e9-e48b21650109 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.000392 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2150-d9a2-4468-a8e9-e48b21650109 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.007871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050468d-5416-4325-a717-ea51f6b9b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.008494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050468d-5416-4325-a717-ea51f6b9b76f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.008779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050468d-5416-4325-a717-ea51f6b9b76f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.009053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050468d-5416-4325-a717-ea51f6b9b76f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.009751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050468d-5416-4325-a717-ea51f6b9b76f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:41.010056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050468d-5416-4325-a717-ea51f6b9b76f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.011535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050468d-5416-4325-a717-ea51f6b9b76f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.022746 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd2150-d9a2-4468-a8e9-e48b21650109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:41.031332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 341/669] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:41.031332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050468d-5416-4325-a717-ea51f6b9b76f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-fccd2150-d9a2-4468-a8e9-e48b21650109 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:41.031571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:41.032548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 329/670] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:41.033576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2150-d9a2-4468-a8e9-e48b21650109 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.060963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:41.139494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050468d-5416-4325-a717-ea51f6b9b76f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:41.144505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2150-d9a2-4468-a8e9-e48b21650109 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:41.179060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2c224-5cb4-4583-b7f6-22784be56d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.179684 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2c224-5cb4-4583-b7f6-22784be56d9a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.179967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2c224-5cb4-4583-b7f6-22784be56d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.180406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2c224-5cb4-4583-b7f6-22784be56d9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.181088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2c224-5cb4-4583-b7f6-22784be56d9a None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=548acf58bf9140ec9f1ba126e10deb45) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:41.181402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2c224-5cb4-4583-b7f6-22784be56d9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.181935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bf6e4-2fc6-4c21-90a9-41cfcf586b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.182595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bf6e4-2fc6-4c21-90a9-41cfcf586b86 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.182860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2c224-5cb4-4583-b7f6-22784be56d9a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.182966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bf6e4-2fc6-4c21-90a9-41cfcf586b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.183134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bf6e4-2fc6-4c21-90a9-41cfcf586b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.193946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bf6e4-2fc6-4c21-90a9-41cfcf586b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.194851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 342/671] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/role_assignments?user.id=548acf58bf9140ec9f1ba126e10deb45&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:41.195388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bf6e4-2fc6-4c21-90a9-41cfcf586b86 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.196458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2c224-5cb4-4583-b7f6-22784be56d9a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d45bf6e4-2fc6-4c21-90a9-41cfcf586b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:41.212725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:41.213731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:41.215134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:41.215134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:41.215134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:41.215134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:41.215134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 330/672] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:41.215134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bf6e4-2fc6-4c21-90a9-41cfcf586b86 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.218982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:41.331582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:41.348803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2c224-5cb4-4583-b7f6-22784be56d9a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:41.357532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bf6e4-2fc6-4c21-90a9-41cfcf586b86 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:41.392794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a587f9-cfa2-411c-8961-3d96c9b698e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.394561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a587f9-cfa2-411c-8961-3d96c9b698e9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.394849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a587f9-cfa2-411c-8961-3d96c9b698e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.395128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a587f9-cfa2-411c-8961-3d96c9b698e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.397871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03de142-a2cf-4fb4-a67c-6e70dbcf1d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.398646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03de142-a2cf-4fb4-a67c-6e70dbcf1d42 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.399024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03de142-a2cf-4fb4-a67c-6e70dbcf1d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.399399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03de142-a2cf-4fb4-a67c-6e70dbcf1d42 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.400221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03de142-a2cf-4fb4-a67c-6e70dbcf1d42 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:41.401673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03de142-a2cf-4fb4-a67c-6e70dbcf1d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.403087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03de142-a2cf-4fb4-a67c-6e70dbcf1d42 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.404205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a587f9-cfa2-411c-8961-3d96c9b698e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.406396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a587f9-cfa2-411c-8961-3d96c9b698e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.412610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 331/673] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:41.414114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03de142-a2cf-4fb4-a67c-6e70dbcf1d42 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-41a587f9-cfa2-411c-8961-3d96c9b698e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:41.422884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:41.423852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:41.424744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:41.424744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:41.424744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:41.424744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:41.424744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:41.424744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:41.424744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:41.424744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:41.424744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 343/674] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:41.425709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a587f9-cfa2-411c-8961-3d96c9b698e9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.494354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:41.537056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03de142-a2cf-4fb4-a67c-6e70dbcf1d42 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:41.540904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a587f9-cfa2-411c-8961-3d96c9b698e9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:41.572484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dab69f-4db0-4ddc-86ee-832ae798326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.573351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dab69f-4db0-4ddc-86ee-832ae798326f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.573796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dab69f-4db0-4ddc-86ee-832ae798326f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.574213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dab69f-4db0-4ddc-86ee-832ae798326f None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.575071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dab69f-4db0-4ddc-86ee-832ae798326f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:41.576062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dab69f-4db0-4ddc-86ee-832ae798326f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.576255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e787aa-3b5f-4b9b-9d8e-3f0c5ca21156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.576900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e787aa-3b5f-4b9b-9d8e-3f0c5ca21156 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.577175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e787aa-3b5f-4b9b-9d8e-3f0c5ca21156 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.577445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e787aa-3b5f-4b9b-9d8e-3f0c5ca21156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.578202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dab69f-4db0-4ddc-86ee-832ae798326f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.588259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e787aa-3b5f-4b9b-9d8e-3f0c5ca21156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.589713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e787aa-3b5f-4b9b-9d8e-3f0c5ca21156 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.592988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 332/675] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:41.593878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dab69f-4db0-4ddc-86ee-832ae798326f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.603195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f2e787aa-3b5f-4b9b-9d8e-3f0c5ca21156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:41.611622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:41.612692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:41.613801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:41.613801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:41.613801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:41.613801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:41.619706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 344/676] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:41.620986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c3990-05b5-46e9-9b24-2a76c3012c97 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.621302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c3990-05b5-46e9-9b24-2a76c3012c97 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.621557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c3990-05b5-46e9-9b24-2a76c3012c97 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.623464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 345/677] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:41.627946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c3990-05b5-46e9-9b24-2a76c3012c97 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.703924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dab69f-4db0-4ddc-86ee-832ae798326f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:41.729370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:41.742838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9018a2a-37fe-4fc5-9da2-260f991800fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.743324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c3990-05b5-46e9-9b24-2a76c3012c97 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:41.743709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9018a2a-37fe-4fc5-9da2-260f991800fe None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.743982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9018a2a-37fe-4fc5-9da2-260f991800fe None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.744251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9018a2a-37fe-4fc5-9da2-260f991800fe None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.745052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9018a2a-37fe-4fc5-9da2-260f991800fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:41.745295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9018a2a-37fe-4fc5-9da2-260f991800fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.746731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9018a2a-37fe-4fc5-9da2-260f991800fe None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.759279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 333/678] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:41.759968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9018a2a-37fe-4fc5-9da2-260f991800fe None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.784736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e003b-1460-451e-9ac4-48c4c8cd519e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.787110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e003b-1460-451e-9ac4-48c4c8cd519e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.787110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e003b-1460-451e-9ac4-48c4c8cd519e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.787110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e003b-1460-451e-9ac4-48c4c8cd519e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.796131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e003b-1460-451e-9ac4-48c4c8cd519e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.797588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e003b-1460-451e-9ac4-48c4c8cd519e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-023e003b-1460-451e-9ac4-48c4c8cd519e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:41.809072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:41.810258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:41.811617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:41.811617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:41.811617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:41.811617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:41.811617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:41.811617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:41.811617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:41.811617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:41.811617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 346/679] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:41.812741 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76f65e-392f-4746-866f-a7f3faeccc8d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.813030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76f65e-392f-4746-866f-a7f3faeccc8d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.813305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76f65e-392f-4746-866f-a7f3faeccc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.832995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7a76f65e-392f-4746-866f-a7f3faeccc8d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:41.834190 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7a76f65e-392f-4746-866f-a7f3faeccc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:41.876260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9018a2a-37fe-4fc5-9da2-260f991800fe None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:41.897891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:41.911865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0a13-b5f4-4e0d-b171-4b456bcf636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:41.912509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0a13-b5f4-4e0d-b171-4b456bcf636e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:41.914476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0a13-b5f4-4e0d-b171-4b456bcf636e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:41.914476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0a13-b5f4-4e0d-b171-4b456bcf636e None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:41.914476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0a13-b5f4-4e0d-b171-4b456bcf636e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:41.914476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0a13-b5f4-4e0d-b171-4b456bcf636e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:41.915490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0a13-b5f4-4e0d-b171-4b456bcf636e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:41.918747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 347/680] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:41.921015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76f65e-392f-4746-866f-a7f3faeccc8d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.929116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 334/681] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:41.929987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0a13-b5f4-4e0d-b171-4b456bcf636e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:41.979712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:42.041816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0a13-b5f4-4e0d-b171-4b456bcf636e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:42.042210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76f65e-392f-4746-866f-a7f3faeccc8d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:42.080248 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb23ff5-c8a6-44a2-b40b-28ab149a3bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:42.080864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb23ff5-c8a6-44a2-b40b-28ab149a3bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:42.081142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb23ff5-c8a6-44a2-b40b-28ab149a3bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:42.081414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb23ff5-c8a6-44a2-b40b-28ab149a3bc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:42.081768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020f45-5650-410e-8d93-db7c0f835ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:42.082167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb23ff5-c8a6-44a2-b40b-28ab149a3bc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:42.082522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb23ff5-c8a6-44a2-b40b-28ab149a3bc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:42.082727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020f45-5650-410e-8d93-db7c0f835ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:42.083107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020f45-5650-410e-8d93-db7c0f835ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:42.083479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020f45-5650-410e-8d93-db7c0f835ef7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:42.083919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb23ff5-c8a6-44a2-b40b-28ab149a3bc3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:42.084479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020f45-5650-410e-8d93-db7c0f835ef7 None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=a0d2552bd0e244dea55eb1eecf471fef) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:42.084888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020f45-5650-410e-8d93-db7c0f835ef7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:42.086407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020f45-5650-410e-8d93-db7c0f835ef7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:42.092747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 348/682] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/role_assignments?user.id=a0d2552bd0e244dea55eb1eecf471fef&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:42.093636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020f45-5650-410e-8d93-db7c0f835ef7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:42.108733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 335/683] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:42.109914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb23ff5-c8a6-44a2-b40b-28ab149a3bc3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:42.165406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:42.219292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020f45-5650-410e-8d93-db7c0f835ef7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:42.261240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb23ff5-c8a6-44a2-b40b-28ab149a3bc3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:42.265895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:42.274630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f9b7f-9585-4832-a01d-84a86ae9a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:42.274630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f9b7f-9585-4832-a01d-84a86ae9a274 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:42.274630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f9b7f-9585-4832-a01d-84a86ae9a274 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:42.274630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f9b7f-9585-4832-a01d-84a86ae9a274 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:42.287978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f9b7f-9585-4832-a01d-84a86ae9a274 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:42.289458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f9b7f-9585-4832-a01d-84a86ae9a274 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3d8f9b7f-9585-4832-a01d-84a86ae9a274 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:42.304561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:42.305573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:42.306639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 349/684] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:42.307295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f9b7f-9585-4832-a01d-84a86ae9a274 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:42.313222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05de9db-3672-480a-99f7-f233ae8022d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:42.313875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05de9db-3672-480a-99f7-f233ae8022d8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:42.314151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05de9db-3672-480a-99f7-f233ae8022d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:42.314421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05de9db-3672-480a-99f7-f233ae8022d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:42.332057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05de9db-3672-480a-99f7-f233ae8022d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f05de9db-3672-480a-99f7-f233ae8022d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:42.339271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:42.340237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:42.341708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 336/685] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:42.343165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05de9db-3672-480a-99f7-f233ae8022d8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:42.432334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f9b7f-9585-4832-a01d-84a86ae9a274 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:42.449959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:42.474106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05de9db-3672-480a-99f7-f233ae8022d8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:42.474665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e247aab-2bf7-483f-932f-744ebdb008a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:42.475298 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e247aab-2bf7-483f-932f-744ebdb008a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:42.475580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e247aab-2bf7-483f-932f-744ebdb008a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:42.475861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e247aab-2bf7-483f-932f-744ebdb008a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:42.476902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e247aab-2bf7-483f-932f-744ebdb008a2 None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=426be56424504e958402d785a068d2f8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:42.477222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e247aab-2bf7-483f-932f-744ebdb008a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:42.478716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e247aab-2bf7-483f-932f-744ebdb008a2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:42.486496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 350/686] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:42 2022] GET /identity/v3/role_assignments?user.id=426be56424504e958402d785a068d2f8&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:42.487307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e247aab-2bf7-483f-932f-744ebdb008a2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:42.522058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29adbb-48f8-4f75-96b1-14c6039c3498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:42.523398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29adbb-48f8-4f75-96b1-14c6039c3498 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:42.523923 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29adbb-48f8-4f75-96b1-14c6039c3498 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:42.524363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29adbb-48f8-4f75-96b1-14c6039c3498 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:42.525516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29adbb-48f8-4f75-96b1-14c6039c3498 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:42.526296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29adbb-48f8-4f75-96b1-14c6039c3498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:42.529139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29adbb-48f8-4f75-96b1-14c6039c3498 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:42.567850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 337/687] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Jul 30 04:45:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:42.573956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29adbb-48f8-4f75-96b1-14c6039c3498 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:42.623212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:42.671558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e247aab-2bf7-483f-932f-744ebdb008a2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:42.722490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc5fa8-739a-4239-832f-db54588dfcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:42.723166 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc5fa8-739a-4239-832f-db54588dfcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:42.723449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc5fa8-739a-4239-832f-db54588dfcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:42.723994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc5fa8-739a-4239-832f-db54588dfcd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:42.725219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc5fa8-739a-4239-832f-db54588dfcd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:42.725533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc5fa8-739a-4239-832f-db54588dfcd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:42.727533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc5fa8-739a-4239-832f-db54588dfcd6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:42.739165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 351/688] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:42.744672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc5fa8-739a-4239-832f-db54588dfcd6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:42.752225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29adbb-48f8-4f75-96b1-14c6039c3498 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:42.792575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c202686-c05a-49dd-b312-677bc43e033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:42.793138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c202686-c05a-49dd-b312-677bc43e033a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:42.793415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c202686-c05a-49dd-b312-677bc43e033a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:42.793686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c202686-c05a-49dd-b312-677bc43e033a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:42.877134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c202686-c05a-49dd-b312-677bc43e033a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:42.878652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c202686-c05a-49dd-b312-677bc43e033a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:42.911421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:42.927406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:43.034747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 338/689] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 30 04:45:43.043816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c202686-c05a-49dd-b312-677bc43e033a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:43.058663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc5fa8-739a-4239-832f-db54588dfcd6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:43.130028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29727-3bed-4a8d-8ae4-fac1c167acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:43.130697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29727-3bed-4a8d-8ae4-fac1c167acbe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.130974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29727-3bed-4a8d-8ae4-fac1c167acbe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.131243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29727-3bed-4a8d-8ae4-fac1c167acbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:43.150528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e29727-3bed-4a8d-8ae4-fac1c167acbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:43.152259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e29727-3bed-4a8d-8ae4-fac1c167acbe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-23e29727-3bed-4a8d-8ae4-fac1c167acbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:43.178565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:43.179518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:43.180439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:43.180439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:43.180439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:43.180439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:43.182234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 352/690] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:43.191410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29727-3bed-4a8d-8ae4-fac1c167acbe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:43.218575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c202686-c05a-49dd-b312-677bc43e033a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:43.243108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:43.261458 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680746e3-b5ac-46a7-999d-e0ce40d5d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:43.262207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680746e3-b5ac-46a7-999d-e0ce40d5d23e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.262373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680746e3-b5ac-46a7-999d-e0ce40d5d23e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.263876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680746e3-b5ac-46a7-999d-e0ce40d5d23e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:43.286260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680746e3-b5ac-46a7-999d-e0ce40d5d23e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:43.287751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680746e3-b5ac-46a7-999d-e0ce40d5d23e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-680746e3-b5ac-46a7-999d-e0ce40d5d23e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:43.315395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:43.316335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:43.317232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:43.317232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:43.317232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:43.317232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:43.317232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 339/691] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:43.325121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680746e3-b5ac-46a7-999d-e0ce40d5d23e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:43.408484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:43.429270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29727-3bed-4a8d-8ae4-fac1c167acbe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:43.450885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680746e3-b5ac-46a7-999d-e0ce40d5d23e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:43.493012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8585d-f015-41b3-b3aa-51aca88b22e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:43.493633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8585d-f015-41b3-b3aa-51aca88b22e4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.493912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8585d-f015-41b3-b3aa-51aca88b22e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.494182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8585d-f015-41b3-b3aa-51aca88b22e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:43.494352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd3c28-0bf0-461a-8c8b-53878a1cbc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:43.494906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8585d-f015-41b3-b3aa-51aca88b22e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:43.495082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd3c28-0bf0-461a-8c8b-53878a1cbc59 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.495241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8585d-f015-41b3-b3aa-51aca88b22e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:43.495390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd3c28-0bf0-461a-8c8b-53878a1cbc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.495565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd3c28-0bf0-461a-8c8b-53878a1cbc59 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:43.496270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd3c28-0bf0-461a-8c8b-53878a1cbc59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:43.496561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8585d-f015-41b3-b3aa-51aca88b22e4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:43.496726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd3c28-0bf0-461a-8c8b-53878a1cbc59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:43.499152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd3c28-0bf0-461a-8c8b-53878a1cbc59 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:43.514200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 353/692] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:43.515581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770d302-dcf9-49e8-82a9-c62db509152b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.515873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770d302-dcf9-49e8-82a9-c62db509152b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.516155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770d302-dcf9-49e8-82a9-c62db509152b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:43.517853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 340/693] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:43.519354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8585d-f015-41b3-b3aa-51aca88b22e4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:43.522175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 354/694] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:43.523188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770d302-dcf9-49e8-82a9-c62db509152b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:43.568138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:43.639289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770d302-dcf9-49e8-82a9-c62db509152b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:43.643380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8585d-f015-41b3-b3aa-51aca88b22e4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:43.680881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882edb61-69eb-47c6-a127-50974f6d1042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:43.681220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51427d-a1c8-439b-9c05-43da9167b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:43.681522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882edb61-69eb-47c6-a127-50974f6d1042 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.681800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882edb61-69eb-47c6-a127-50974f6d1042 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.682006 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51427d-a1c8-439b-9c05-43da9167b702 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.682171 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882edb61-69eb-47c6-a127-50974f6d1042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:43.682323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51427d-a1c8-439b-9c05-43da9167b702 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.682503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51427d-a1c8-439b-9c05-43da9167b702 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:43.689975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882edb61-69eb-47c6-a127-50974f6d1042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:43.691609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882edb61-69eb-47c6-a127-50974f6d1042 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:43.691797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51427d-a1c8-439b-9c05-43da9167b702 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:43.693261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51427d-a1c8-439b-9c05-43da9167b702 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-882edb61-69eb-47c6-a127-50974f6d1042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:43.703637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:43.705411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:43.707609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:43.707609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:43.707609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:43.707609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:43.707609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:43.707609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:43.707609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:43.707609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ac51427d-a1c8-439b-9c05-43da9167b702 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:43.708630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:43.710703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:43.719830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:43.720531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 341/695] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:43.720531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d382165-a636-454c-9a9c-c0e99b7a0d08 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.720531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d382165-a636-454c-9a9c-c0e99b7a0d08 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.720531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d382165-a636-454c-9a9c-c0e99b7a0d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:43.720860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 355/696] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Jul 30 04:45:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:43.720860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51427d-a1c8-439b-9c05-43da9167b702 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:43.732112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:43.732417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9d382165-a636-454c-9a9c-c0e99b7a0d08 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:43.734667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9d382165-a636-454c-9a9c-c0e99b7a0d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:43.838535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 342/697] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:43.838535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d382165-a636-454c-9a9c-c0e99b7a0d08 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:43.839972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51427d-a1c8-439b-9c05-43da9167b702 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:43.862227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:43.906472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c4e5c-9c36-48a1-a329-43e746e39dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:43.906472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c4e5c-9c36-48a1-a329-43e746e39dfe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.906472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c4e5c-9c36-48a1-a329-43e746e39dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.906472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c4e5c-9c36-48a1-a329-43e746e39dfe None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:43.906907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c4e5c-9c36-48a1-a329-43e746e39dfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:43.906907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c4e5c-9c36-48a1-a329-43e746e39dfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:43.907267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c4e5c-9c36-48a1-a329-43e746e39dfe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:43.924124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 356/698] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:43.924993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c4e5c-9c36-48a1-a329-43e746e39dfe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:43.958702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d382165-a636-454c-9a9c-c0e99b7a0d08 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:43.997526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2170d4c-12d2-4d50-b893-3b618339286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:43.998256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2170d4c-12d2-4d50-b893-3b618339286d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:43.999100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2170d4c-12d2-4d50-b893-3b618339286d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:43.999381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2170d4c-12d2-4d50-b893-3b618339286d None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.000470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2170d4c-12d2-4d50-b893-3b618339286d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:44.000775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2170d4c-12d2-4d50-b893-3b618339286d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:44.002935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2170d4c-12d2-4d50-b893-3b618339286d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:44.018181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:44.031059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 343/699] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Jul 30 04:45:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:44.032985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2170d4c-12d2-4d50-b893-3b618339286d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:44.052183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c4e5c-9c36-48a1-a329-43e746e39dfe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:44.089930 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb2484-90a8-47a0-8f0c-4a5b63c425e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:44.090314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb2484-90a8-47a0-8f0c-4a5b63c425e6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.090641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb2484-90a8-47a0-8f0c-4a5b63c425e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.090943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb2484-90a8-47a0-8f0c-4a5b63c425e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.102081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb2484-90a8-47a0-8f0c-4a5b63c425e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:44.103800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb2484-90a8-47a0-8f0c-4a5b63c425e6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e0bb2484-90a8-47a0-8f0c-4a5b63c425e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:44.119553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:44.120550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:44.121742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:44.121742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:44.121742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:44.121742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:44.121742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 357/700] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:44.123284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb2484-90a8-47a0-8f0c-4a5b63c425e6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:44.148201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:44.174940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2170d4c-12d2-4d50-b893-3b618339286d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:44.216252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5d5cd-5b29-4f77-a816-e631730574b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:44.217667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5d5cd-5b29-4f77-a816-e631730574b3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.218475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5d5cd-5b29-4f77-a816-e631730574b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.218966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5d5cd-5b29-4f77-a816-e631730574b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.220251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5d5cd-5b29-4f77-a816-e631730574b3 None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=54bda2af62ee4ee39a9a350b7ce6430e) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:44.221236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5d5cd-5b29-4f77-a816-e631730574b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:44.223465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5d5cd-5b29-4f77-a816-e631730574b3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:44.231975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 344/701] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:44 2022] GET /identity/v3/role_assignments?user.id=54bda2af62ee4ee39a9a350b7ce6430e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:44.233238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5d5cd-5b29-4f77-a816-e631730574b3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:44.253353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb2484-90a8-47a0-8f0c-4a5b63c425e6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:44.310722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:44.345287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e0d60-fc5e-47c7-a206-fe7ab9043902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:44.345597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e0d60-fc5e-47c7-a206-fe7ab9043902 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.345880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e0d60-fc5e-47c7-a206-fe7ab9043902 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.346152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e0d60-fc5e-47c7-a206-fe7ab9043902 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.367544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5d5cd-5b29-4f77-a816-e631730574b3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:44.412806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b350b-8daa-48d9-9027-1074adcc4be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:44.413947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b350b-8daa-48d9-9027-1074adcc4be9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.414701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b350b-8daa-48d9-9027-1074adcc4be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.415180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b350b-8daa-48d9-9027-1074adcc4be9 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.416181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b350b-8daa-48d9-9027-1074adcc4be9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:44.416699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b350b-8daa-48d9-9027-1074adcc4be9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:44.419509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b350b-8daa-48d9-9027-1074adcc4be9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:44.435315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 345/702] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:44.437533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f20826-cf17-4384-82f8-1e54dfc05d41 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.437824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f20826-cf17-4384-82f8-1e54dfc05d41 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.438097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f20826-cf17-4384-82f8-1e54dfc05d41 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.440069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 346/703] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:44.440844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f20826-cf17-4384-82f8-1e54dfc05d41 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:44.453297 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:44.482821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e0d60-fc5e-47c7-a206-fe7ab9043902 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:44.484390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e0d60-fc5e-47c7-a206-fe7ab9043902 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:44.561581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f20826-cf17-4384-82f8-1e54dfc05d41 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:44.602066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b131e41-c11e-40e6-b658-12e500dcb1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:44.602982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b131e41-c11e-40e6-b658-12e500dcb1cf None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.603207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b131e41-c11e-40e6-b658-12e500dcb1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.603631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b131e41-c11e-40e6-b658-12e500dcb1cf None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.604633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b131e41-c11e-40e6-b658-12e500dcb1cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:44.606552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b131e41-c11e-40e6-b658-12e500dcb1cf None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:44.611770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 358/704] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:44.613059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da1d6b-8808-4938-a288-6be9647111d6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.613347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da1d6b-8808-4938-a288-6be9647111d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.613618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da1d6b-8808-4938-a288-6be9647111d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.631830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 347/705] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:45:44 2022] POST /identity/v3/users => generated 254 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:44.632476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b131e41-c11e-40e6-b658-12e500dcb1cf None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:44.635168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a7da1d6b-8808-4938-a288-6be9647111d6 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:44.636538 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7da1d6b-8808-4938-a288-6be9647111d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:44.659075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:44.717870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:44.731303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 359/706] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:44.758079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da1d6b-8808-4938-a288-6be9647111d6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:44.795754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b131e41-c11e-40e6-b658-12e500dcb1cf None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:44.873765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190b343-bd3f-4e07-bd61-f4663c9c9b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:44.874623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190b343-bd3f-4e07-bd61-f4663c9c9b8a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.874906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190b343-bd3f-4e07-bd61-f4663c9c9b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.875180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190b343-bd3f-4e07-bd61-f4663c9c9b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.883819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da1d6b-8808-4938-a288-6be9647111d6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:44.897680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190b343-bd3f-4e07-bd61-f4663c9c9b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:44.899181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190b343-bd3f-4e07-bd61-f4663c9c9b8a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:44.926054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2cb87-1acf-459a-8922-a057380f1930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:44.926742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2cb87-1acf-459a-8922-a057380f1930 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.927017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2cb87-1acf-459a-8922-a057380f1930 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.927285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2cb87-1acf-459a-8922-a057380f1930 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7190b343-bd3f-4e07-bd61-f4663c9c9b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:44.927832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:44.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:44.929610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:44.929610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:44.929610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:44.929610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:44.929843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2cb87-1acf-459a-8922-a057380f1930 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:44.929843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2cb87-1acf-459a-8922-a057380f1930 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:44.929979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 348/707] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:44.934459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91ab55-0a6c-4782-984f-71763195cd1d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.934459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91ab55-0a6c-4782-984f-71763195cd1d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.934459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91ab55-0a6c-4782-984f-71763195cd1d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.934459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 349/708] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:44.939536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e17225-45f8-4d64-8586-d063e1b73637 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:44.939818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e17225-45f8-4d64-8586-d063e1b73637 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:44.940152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e17225-45f8-4d64-8586-d063e1b73637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eec2cb87-1acf-459a-8922-a057380f1930 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:44.941201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:44.943382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 30 04:45:44.945360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:44.947251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 360/709] 10.222.0.20 () {56 vars in 1129 bytes} [Sat Jul 30 04:45:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:44.948129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2cb87-1acf-459a-8922-a057380f1930 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:44.968388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-24e17225-45f8-4d64-8586-d063e1b73637 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:44.969610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-24e17225-45f8-4d64-8586-d063e1b73637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:44.979767 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:45.064434 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 350/710] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:45.065327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e17225-45f8-4d64-8586-d063e1b73637 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:45.096912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2cb87-1acf-459a-8922-a057380f1930 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:45.116050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:45.144131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dc88a-df89-4547-8ad7-a995d4979c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:45.145553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dc88a-df89-4547-8ad7-a995d4979c5b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:45.145838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dc88a-df89-4547-8ad7-a995d4979c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:45.146113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dc88a-df89-4547-8ad7-a995d4979c5b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:45.146849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dc88a-df89-4547-8ad7-a995d4979c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:45.147155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dc88a-df89-4547-8ad7-a995d4979c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:45.148536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0dc88a-df89-4547-8ad7-a995d4979c5b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:45.166269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 361/711] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:45.168066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dc88a-df89-4547-8ad7-a995d4979c5b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:45.177925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e17225-45f8-4d64-8586-d063e1b73637 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:45.219688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d9e28-ad35-4244-8d76-9e8bbbb39429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:45.220315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d9e28-ad35-4244-8d76-9e8bbbb39429 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:45.220595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d9e28-ad35-4244-8d76-9e8bbbb39429 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:45.220960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d9e28-ad35-4244-8d76-9e8bbbb39429 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:45.221646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d9e28-ad35-4244-8d76-9e8bbbb39429 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:45.221947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d9e28-ad35-4244-8d76-9e8bbbb39429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:45.223355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d9e28-ad35-4244-8d76-9e8bbbb39429 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:45.228376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 351/712] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Jul 30 04:45:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:45.229203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d9e28-ad35-4244-8d76-9e8bbbb39429 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:45.271382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:45.297801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dc88a-df89-4547-8ad7-a995d4979c5b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:45.335785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ab4e-0674-491a-8857-a421ba0d7af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:45.336476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ab4e-0674-491a-8857-a421ba0d7af2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:45.336712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ab4e-0674-491a-8857-a421ba0d7af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:45.336987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ab4e-0674-491a-8857-a421ba0d7af2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:45.337625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ab4e-0674-491a-8857-a421ba0d7af2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:45.339149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ab4e-0674-491a-8857-a421ba0d7af2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:45.346380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d9e28-ad35-4244-8d76-9e8bbbb39429 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5746ab4e-0674-491a-8857-a421ba0d7af2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:45.349639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:45.350648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 30 04:45:45.351490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:45.354716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 362/713] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:45 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:45.355548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ab4e-0674-491a-8857-a421ba0d7af2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:45.362099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:45.389843 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012ae22-0097-45cd-be06-ead8a4b503d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:45.391468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012ae22-0097-45cd-be06-ead8a4b503d4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:45.391965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012ae22-0097-45cd-be06-ead8a4b503d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:45.392469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012ae22-0097-45cd-be06-ead8a4b503d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:45.400578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012ae22-0097-45cd-be06-ead8a4b503d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:45.402574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012ae22-0097-45cd-be06-ead8a4b503d4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f012ae22-0097-45cd-be06-ead8a4b503d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:45.415882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:45.416809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:45.417721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:45.417721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:45.417721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:45.417721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:45.417721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:45.417721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:45.417721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:45.417721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:45.417721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 352/714] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:45.418219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012ae22-0097-45cd-be06-ead8a4b503d4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:45.479559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ab4e-0674-491a-8857-a421ba0d7af2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:45.519333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b9e2a-c559-4f2b-b0c4-1c332d1d25e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:45.519962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b9e2a-c559-4f2b-b0c4-1c332d1d25e4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:45.520246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b9e2a-c559-4f2b-b0c4-1c332d1d25e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:45.520523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b9e2a-c559-4f2b-b0c4-1c332d1d25e4 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:45.521208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b9e2a-c559-4f2b-b0c4-1c332d1d25e4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:45.521515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b9e2a-c559-4f2b-b0c4-1c332d1d25e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:45.521734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012ae22-0097-45cd-be06-ead8a4b503d4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:45.522933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b9e2a-c559-4f2b-b0c4-1c332d1d25e4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:45.530244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 363/715] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Jul 30 04:45:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:45.531170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b9e2a-c559-4f2b-b0c4-1c332d1d25e4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:45.547461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:45.559808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952421b-e0b8-45f1-a836-e13001dda6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:45.560447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952421b-e0b8-45f1-a836-e13001dda6db None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:45.560722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952421b-e0b8-45f1-a836-e13001dda6db None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:45.560994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952421b-e0b8-45f1-a836-e13001dda6db None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:45.561688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952421b-e0b8-45f1-a836-e13001dda6db None admin] RBAC: Adding query filter params (name=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:45.561988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952421b-e0b8-45f1-a836-e13001dda6db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:45.563439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952421b-e0b8-45f1-a836-e13001dda6db None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:45.573267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 353/716] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:45:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:45.574651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952421b-e0b8-45f1-a836-e13001dda6db None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:45.608226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:45.651988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b9e2a-c559-4f2b-b0c4-1c332d1d25e4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:45.691637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3086446-10d1-48b9-9915-1794e8b87b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:45.692305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3086446-10d1-48b9-9915-1794e8b87b44 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:45.692733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3086446-10d1-48b9-9915-1794e8b87b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:45.693020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3086446-10d1-48b9-9915-1794e8b87b44 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:45.693789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3086446-10d1-48b9-9915-1794e8b87b44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:45.694100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3086446-10d1-48b9-9915-1794e8b87b44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:45.695744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3086446-10d1-48b9-9915-1794e8b87b44 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:45.711522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 364/717] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:45.719023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3086446-10d1-48b9-9915-1794e8b87b44 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:45.747876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952421b-e0b8-45f1-a836-e13001dda6db None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:45.802846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:45.813102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75947d1-d038-404f-9b37-6e9377092e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:45.813738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75947d1-d038-404f-9b37-6e9377092e84 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:45.814014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75947d1-d038-404f-9b37-6e9377092e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:45.814288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75947d1-d038-404f-9b37-6e9377092e84 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:45.815061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75947d1-d038-404f-9b37-6e9377092e84 None admin] RBAC: Adding query filter params (type=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:45.815364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75947d1-d038-404f-9b37-6e9377092e84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:45.816750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75947d1-d038-404f-9b37-6e9377092e84 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:45.828610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 354/718] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:45:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:45.842476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3086446-10d1-48b9-9915-1794e8b87b44 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:45.882823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4238b7-2320-4b0b-8238-846be7c824c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:45.883477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4238b7-2320-4b0b-8238-846be7c824c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:45.883759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4238b7-2320-4b0b-8238-846be7c824c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:45.884033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4238b7-2320-4b0b-8238-846be7c824c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:45.884840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4238b7-2320-4b0b-8238-846be7c824c9 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=a1dc3ad467d9496cb3e0086a0cd8215e) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:45.885151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4238b7-2320-4b0b-8238-846be7c824c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:45.886629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4238b7-2320-4b0b-8238-846be7c824c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:45.894143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 365/719] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:45 2022] GET /identity/v3/role_assignments?user.id=a1dc3ad467d9496cb3e0086a0cd8215e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:46.777714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dc71d-eece-4c8e-a294-efc08c2cf8f4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:46.778840 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dc71d-eece-4c8e-a294-efc08c2cf8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:46.778959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dc71d-eece-4c8e-a294-efc08c2cf8f4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:46.780512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 355/720] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:46.785961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4867-3b99-4244-a34d-64361086bfca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:46.786420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4867-3b99-4244-a34d-64361086bfca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:46.786525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b4867-3b99-4244-a34d-64361086bfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:46.811652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-411b4867-3b99-4244-a34d-64361086bfca None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:46.812879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-411b4867-3b99-4244-a34d-64361086bfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:46.900981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 366/721] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:46.939706 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dc71d-eece-4c8e-a294-efc08c2cf8f4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:47.088973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:47.099927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dc71d-eece-4c8e-a294-efc08c2cf8f4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:47.149913 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23737ab-28aa-4e10-9c56-832d5df9710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:47.150581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23737ab-28aa-4e10-9c56-832d5df9710d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:47.150867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23737ab-28aa-4e10-9c56-832d5df9710d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:47.151141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23737ab-28aa-4e10-9c56-832d5df9710d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:47.167023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97400db8-c7c8-4c9d-b46f-4e5e421c22a4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:47.167395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97400db8-c7c8-4c9d-b46f-4e5e421c22a4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:47.167574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97400db8-c7c8-4c9d-b46f-4e5e421c22a4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:47.168216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23737ab-28aa-4e10-9c56-832d5df9710d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:47.169199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 367/722] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a23737ab-28aa-4e10-9c56-832d5df9710d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:47.173986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:47.175048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:47.175931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 356/723] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:47.181386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540dfa4-c1d5-4754-9645-ef40299c7b1c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:47.181558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23737ab-28aa-4e10-9c56-832d5df9710d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:47.181651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540dfa4-c1d5-4754-9645-ef40299c7b1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:47.181928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540dfa4-c1d5-4754-9645-ef40299c7b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:47.221331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a540dfa4-c1d5-4754-9645-ef40299c7b1c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:47.224232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a540dfa4-c1d5-4754-9645-ef40299c7b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:47.342811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23737ab-28aa-4e10-9c56-832d5df9710d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:47.345890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:47.368999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 368/724] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:47.396356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540dfa4-c1d5-4754-9645-ef40299c7b1c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:47.422935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da471497-561d-498e-80e1-59a0f36050de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:47.424981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da471497-561d-498e-80e1-59a0f36050de None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:47.425495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da471497-561d-498e-80e1-59a0f36050de None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:47.426927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da471497-561d-498e-80e1-59a0f36050de None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:47.427881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da471497-561d-498e-80e1-59a0f36050de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:47.431448 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da471497-561d-498e-80e1-59a0f36050de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:47.433057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da471497-561d-498e-80e1-59a0f36050de None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:47.457604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 357/725] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:47.475926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da471497-561d-498e-80e1-59a0f36050de None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:47.502915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:47.569837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540dfa4-c1d5-4754-9645-ef40299c7b1c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:47.606817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba23e8-7ad0-4968-8e60-049a6e4aff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:47.607789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba23e8-7ad0-4968-8e60-049a6e4aff3c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:47.608368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba23e8-7ad0-4968-8e60-049a6e4aff3c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:47.608896 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba23e8-7ad0-4968-8e60-049a6e4aff3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:47.619973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba23e8-7ad0-4968-8e60-049a6e4aff3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-87ba23e8-7ad0-4968-8e60-049a6e4aff3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:47.625569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:47.626616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:47.627485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:47.630199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 369/726] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:47.636983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba23e8-7ad0-4968-8e60-049a6e4aff3c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:47.658523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da471497-561d-498e-80e1-59a0f36050de None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:47.701481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a8a95-7ceb-49d5-bdae-0627fcd93d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:47.703469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a8a95-7ceb-49d5-bdae-0627fcd93d98 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:47.704695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a8a95-7ceb-49d5-bdae-0627fcd93d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:47.705306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a8a95-7ceb-49d5-bdae-0627fcd93d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:47.757071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:47.860678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a8a95-7ceb-49d5-bdae-0627fcd93d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:47.862077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a8a95-7ceb-49d5-bdae-0627fcd93d98 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4b0a8a95-7ceb-49d5-bdae-0627fcd93d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:47.877449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:47.878417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:47.879582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:47.879582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:47.879582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:47.879582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:47.879582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 358/727] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:47.886714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a8a95-7ceb-49d5-bdae-0627fcd93d98 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:47.902558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba23e8-7ad0-4968-8e60-049a6e4aff3c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:47.939135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a3857-5384-4325-9825-1b438ca7e18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:47.939863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a3857-5384-4325-9825-1b438ca7e18d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:47.940142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a3857-5384-4325-9825-1b438ca7e18d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:47.940413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a3857-5384-4325-9825-1b438ca7e18d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:47.941128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a3857-5384-4325-9825-1b438ca7e18d None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:47.941426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a3857-5384-4325-9825-1b438ca7e18d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:47.943026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a3857-5384-4325-9825-1b438ca7e18d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:47.951994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 370/728] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:47.958357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a3857-5384-4325-9825-1b438ca7e18d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:47.971233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:48.050790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a8a95-7ceb-49d5-bdae-0627fcd93d98 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:48.101747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a3857-5384-4325-9825-1b438ca7e18d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:48.120071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825dae7-8cdf-4c25-86ef-f8ab117eee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:48.120613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825dae7-8cdf-4c25-86ef-f8ab117eee89 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.120885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825dae7-8cdf-4c25-86ef-f8ab117eee89 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.121149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825dae7-8cdf-4c25-86ef-f8ab117eee89 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.122427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825dae7-8cdf-4c25-86ef-f8ab117eee89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:48.122741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825dae7-8cdf-4c25-86ef-f8ab117eee89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:48.124887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825dae7-8cdf-4c25-86ef-f8ab117eee89 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:48.128423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:48.144449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 359/729] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:48.151871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825dae7-8cdf-4c25-86ef-f8ab117eee89 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:48.170637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c8268-b535-451e-aed2-3dc98cb3b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:48.171282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c8268-b535-451e-aed2-3dc98cb3b05a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.171559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c8268-b535-451e-aed2-3dc98cb3b05a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.171829 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c8268-b535-451e-aed2-3dc98cb3b05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.194225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c8268-b535-451e-aed2-3dc98cb3b05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:48.196823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c8268-b535-451e-aed2-3dc98cb3b05a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-877c8268-b535-451e-aed2-3dc98cb3b05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:48.229061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:48.230017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:48.231107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:48.231107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:48.231107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:48.231107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:48.231107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 371/730] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:48.238104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c8268-b535-451e-aed2-3dc98cb3b05a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:48.319376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825dae7-8cdf-4c25-86ef-f8ab117eee89 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:48.359914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:48.363369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf79bd-0d7a-4ecd-aae6-4d2e7386c42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:48.363940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf79bd-0d7a-4ecd-aae6-4d2e7386c42f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.364259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf79bd-0d7a-4ecd-aae6-4d2e7386c42f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.364708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf79bd-0d7a-4ecd-aae6-4d2e7386c42f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.374763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf79bd-0d7a-4ecd-aae6-4d2e7386c42f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:48.376438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf79bd-0d7a-4ecd-aae6-4d2e7386c42f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-04bf79bd-0d7a-4ecd-aae6-4d2e7386c42f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:48.390417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:48.391628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:48.392933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 360/731] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:48.393670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f916-3049-4f9f-a3fc-a8b11ea0ae8e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.393670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f916-3049-4f9f-a3fc-a8b11ea0ae8e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.393837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987f916-3049-4f9f-a3fc-a8b11ea0ae8e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.398838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c8268-b535-451e-aed2-3dc98cb3b05a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:48.399096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 361/732] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:48.400125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db13249-cab7-432c-a01f-517a88dde9fa None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.400411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db13249-cab7-432c-a01f-517a88dde9fa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.400684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db13249-cab7-432c-a01f-517a88dde9fa None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.402317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 362/733] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:48.402875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db13249-cab7-432c-a01f-517a88dde9fa None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:48.441300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0b46e-cecb-4121-b927-2061aa64f2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:48.441926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0b46e-cecb-4121-b927-2061aa64f2ab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.442233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0b46e-cecb-4121-b927-2061aa64f2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.442545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0b46e-cecb-4121-b927-2061aa64f2ab None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.443229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0b46e-cecb-4121-b927-2061aa64f2ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:48.443537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0b46e-cecb-4121-b927-2061aa64f2ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:48.444911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0b46e-cecb-4121-b927-2061aa64f2ab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:48.457648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 372/734] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:48.458969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb2222-10db-4271-9d2e-4fcff57d4e2c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.459254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb2222-10db-4271-9d2e-4fcff57d4e2c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.459527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb2222-10db-4271-9d2e-4fcff57d4e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.478088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8ebb2222-10db-4271-9d2e-4fcff57d4e2c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:48.479307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ebb2222-10db-4271-9d2e-4fcff57d4e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:48.502337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:48.516487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db13249-cab7-432c-a01f-517a88dde9fa None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:48.550947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb13dd8-30c3-4218-a115-77cb29de1c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:48.551551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb13dd8-30c3-4218-a115-77cb29de1c48 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.551824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb13dd8-30c3-4218-a115-77cb29de1c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.552091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb13dd8-30c3-4218-a115-77cb29de1c48 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.552789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 373/735] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:48.552883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb13dd8-30c3-4218-a115-77cb29de1c48 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:48.553157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb13dd8-30c3-4218-a115-77cb29de1c48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:48.554031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff5a4c-fb76-4c57-8b3f-33c4cc69c98e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.554317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff5a4c-fb76-4c57-8b3f-33c4cc69c98e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.554610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb13dd8-30c3-4218-a115-77cb29de1c48 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:48.554708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff5a4c-fb76-4c57-8b3f-33c4cc69c98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.571460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cfff5a4c-fb76-4c57-8b3f-33c4cc69c98e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:48.572633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfff5a4c-fb76-4c57-8b3f-33c4cc69c98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:48.579639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 363/736] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Jul 30 04:45:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:48.580453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb13dd8-30c3-4218-a115-77cb29de1c48 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:48.628831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:48.640456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 374/737] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:48.641870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4c47b-563f-418f-bda2-7ff960f55d9d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.642160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4c47b-563f-418f-bda2-7ff960f55d9d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.642458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4c47b-563f-418f-bda2-7ff960f55d9d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.643712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 375/738] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:48.644397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4c47b-563f-418f-bda2-7ff960f55d9d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:48.689901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb13dd8-30c3-4218-a115-77cb29de1c48 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:48.725340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968d07d-9272-4d6e-b364-d518434d3aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:48.725977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968d07d-9272-4d6e-b364-d518434d3aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.726252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968d07d-9272-4d6e-b364-d518434d3aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.726550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968d07d-9272-4d6e-b364-d518434d3aa5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.731836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:48.734858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968d07d-9272-4d6e-b364-d518434d3aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:48.736297 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968d07d-9272-4d6e-b364-d518434d3aa5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c968d07d-9272-4d6e-b364-d518434d3aa5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:48.749207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:48.750362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:45:48.751398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:48.751946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4c47b-563f-418f-bda2-7ff960f55d9d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:48.754745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 364/739] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:45:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:48.755727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968d07d-9272-4d6e-b364-d518434d3aa5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:48.785726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196e7b1-125b-4441-92ff-e7afdbd0a6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:48.786363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196e7b1-125b-4441-92ff-e7afdbd0a6ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.786666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196e7b1-125b-4441-92ff-e7afdbd0a6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.786935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196e7b1-125b-4441-92ff-e7afdbd0a6ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.797132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196e7b1-125b-4441-92ff-e7afdbd0a6ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8196e7b1-125b-4441-92ff-e7afdbd0a6ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:48.801932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:48.802948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:48.803783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 376/740] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:48.804749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f7f7-93b9-46e5-9a40-d84c48f6bcaa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.804749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f7f7-93b9-46e5-9a40-d84c48f6bcaa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.804749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53f7f7-93b9-46e5-9a40-d84c48f6bcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.822148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fb53f7f7-93b9-46e5-9a40-d84c48f6bcaa None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:48.823364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb53f7f7-93b9-46e5-9a40-d84c48f6bcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:48.857942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968d07d-9272-4d6e-b364-d518434d3aa5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:48.866582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:48.890044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 377/741] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 30 04:45:48.890819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53f7f7-93b9-46e5-9a40-d84c48f6bcaa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:48.892179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1add88-a7f8-4716-8f82-52b1df932346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:48.892791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1add88-a7f8-4716-8f82-52b1df932346 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:48.893067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1add88-a7f8-4716-8f82-52b1df932346 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:48.893333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1add88-a7f8-4716-8f82-52b1df932346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:48.964453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1add88-a7f8-4716-8f82-52b1df932346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:48.965917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1add88-a7f8-4716-8f82-52b1df932346 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:48.968124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:48.994723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53f7f7-93b9-46e5-9a40-d84c48f6bcaa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:49.032140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be4190-168c-4495-8c92-7a436ef90b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:49.032765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be4190-168c-4495-8c92-7a436ef90b54 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:49.033044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be4190-168c-4495-8c92-7a436ef90b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:49.033313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be4190-168c-4495-8c92-7a436ef90b54 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:49.034011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be4190-168c-4495-8c92-7a436ef90b54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:49.035420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be4190-168c-4495-8c92-7a436ef90b54 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:49.051795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 378/742] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:45:48 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:49.052612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be4190-168c-4495-8c92-7a436ef90b54 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:49.072536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 365/743] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 30 04:45:49.072536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1add88-a7f8-4716-8f82-52b1df932346 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:49.083528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:49.183510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be4190-168c-4495-8c92-7a436ef90b54 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:49.190074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1add88-a7f8-4716-8f82-52b1df932346 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:49.218523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:49.225755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bd0b7-a19a-4caf-bb42-0e86bc2ac667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:49.227120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bd0b7-a19a-4caf-bb42-0e86bc2ac667 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:49.227120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bd0b7-a19a-4caf-bb42-0e86bc2ac667 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:49.227406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bd0b7-a19a-4caf-bb42-0e86bc2ac667 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:49.228776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bd0b7-a19a-4caf-bb42-0e86bc2ac667 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:49.228894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bd0b7-a19a-4caf-bb42-0e86bc2ac667 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:49.230278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bd0b7-a19a-4caf-bb42-0e86bc2ac667 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:49.235193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28037cd5-0214-4215-96b8-7f2a8b755952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:49.236108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28037cd5-0214-4215-96b8-7f2a8b755952 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:49.236108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28037cd5-0214-4215-96b8-7f2a8b755952 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:49.236357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28037cd5-0214-4215-96b8-7f2a8b755952 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:49.237702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28037cd5-0214-4215-96b8-7f2a8b755952 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:49.237702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28037cd5-0214-4215-96b8-7f2a8b755952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:49.238744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28037cd5-0214-4215-96b8-7f2a8b755952 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:49.248166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 366/744] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:49.249323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28037cd5-0214-4215-96b8-7f2a8b755952 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:49.253414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 379/745] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Jul 30 04:45:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:49.254240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bd0b7-a19a-4caf-bb42-0e86bc2ac667 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:49.323586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:49.357475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28037cd5-0214-4215-96b8-7f2a8b755952 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:49.364193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bd0b7-a19a-4caf-bb42-0e86bc2ac667 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:49.393839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869cf48-9a24-4444-a6b0-ea5c28c35bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:49.394488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869cf48-9a24-4444-a6b0-ea5c28c35bcd None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:49.394763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869cf48-9a24-4444-a6b0-ea5c28c35bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:49.395036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869cf48-9a24-4444-a6b0-ea5c28c35bcd None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:49.395728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869cf48-9a24-4444-a6b0-ea5c28c35bcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:49.397088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869cf48-9a24-4444-a6b0-ea5c28c35bcd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:49.398475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de22ac0-b8c0-43d8-8cb7-b83b8ba12512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:49.399131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de22ac0-b8c0-43d8-8cb7-b83b8ba12512 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:49.399414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de22ac0-b8c0-43d8-8cb7-b83b8ba12512 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:49.399688 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de22ac0-b8c0-43d8-8cb7-b83b8ba12512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:49.411412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de22ac0-b8c0-43d8-8cb7-b83b8ba12512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:49.412864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de22ac0-b8c0-43d8-8cb7-b83b8ba12512 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:49.419501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 367/746] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:45:49 2022] POST /identity/v3/services => generated 240 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:49.419501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869cf48-9a24-4444-a6b0-ea5c28c35bcd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8de22ac0-b8c0-43d8-8cb7-b83b8ba12512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:49.440359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:49.441303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:49.442218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:49.442218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:49.442218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:49.442218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:49.442218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 380/747] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:49.443374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de22ac0-b8c0-43d8-8cb7-b83b8ba12512 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:49.502662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:49.569934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869cf48-9a24-4444-a6b0-ea5c28c35bcd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:49.575195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de22ac0-b8c0-43d8-8cb7-b83b8ba12512 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:49.605896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:49.620358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f370b-ed5b-4c7d-871d-5f8b6c78e13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:49.621013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f370b-ed5b-4c7d-871d-5f8b6c78e13f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:49.621570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f370b-ed5b-4c7d-871d-5f8b6c78e13f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:49.621570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f370b-ed5b-4c7d-871d-5f8b6c78e13f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:49.624032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ca190-52b5-4be6-8f1d-e130f86fe6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:49.624032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ca190-52b5-4be6-8f1d-e130f86fe6ab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:49.624312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ca190-52b5-4be6-8f1d-e130f86fe6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:49.624582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ca190-52b5-4be6-8f1d-e130f86fe6ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:49.639844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f370b-ed5b-4c7d-871d-5f8b6c78e13f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:49.641662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f370b-ed5b-4c7d-871d-5f8b6c78e13f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-118f370b-ed5b-4c7d-871d-5f8b6c78e13f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:49.660182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:49.661149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:49.662061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:49.662061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:49.662061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:49.662061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:49.664307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 368/748] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:49.665294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f370b-ed5b-4c7d-871d-5f8b6c78e13f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:49.714225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ca190-52b5-4be6-8f1d-e130f86fe6ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:49.716949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ca190-52b5-4be6-8f1d-e130f86fe6ab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:49.777760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f370b-ed5b-4c7d-871d-5f8b6c78e13f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:49.798828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:49.822614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bda58d-8428-454a-b669-f5c24eb9f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:49.823348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bda58d-8428-454a-b669-f5c24eb9f5dc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:49.823703 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bda58d-8428-454a-b669-f5c24eb9f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:49.824042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bda58d-8428-454a-b669-f5c24eb9f5dc None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:49.825294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bda58d-8428-454a-b669-f5c24eb9f5dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:49.825669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bda58d-8428-454a-b669-f5c24eb9f5dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:49.827456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bda58d-8428-454a-b669-f5c24eb9f5dc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:49.846556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 369/749] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:49.847712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bda58d-8428-454a-b669-f5c24eb9f5dc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:49.858607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 381/750] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:45:49.859732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ca190-52b5-4be6-8f1d-e130f86fe6ab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:49.892950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:49.967130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bda58d-8428-454a-b669-f5c24eb9f5dc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:49.985061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ca190-52b5-4be6-8f1d-e130f86fe6ab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:50.017928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75622350-32e3-4f55-acb9-a449970540d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.018891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75622350-32e3-4f55-acb9-a449970540d2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.019354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75622350-32e3-4f55-acb9-a449970540d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.019785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75622350-32e3-4f55-acb9-a449970540d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.023050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75622350-32e3-4f55-acb9-a449970540d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:50.023225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75622350-32e3-4f55-acb9-a449970540d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.024574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75622350-32e3-4f55-acb9-a449970540d2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.039955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 370/751] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:50.040324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed616c-1e5c-4897-b738-5eac086e0d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.040919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75622350-32e3-4f55-acb9-a449970540d2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.041543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed616c-1e5c-4897-b738-5eac086e0d1d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.042018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed616c-1e5c-4897-b738-5eac086e0d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.042561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed616c-1e5c-4897-b738-5eac086e0d1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.052666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed616c-1e5c-4897-b738-5eac086e0d1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.055125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed616c-1e5c-4897-b738-5eac086e0d1d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.059066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ebed616c-1e5c-4897-b738-5eac086e0d1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:50.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:50.075552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:50.076609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:50.076609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:50.076609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:50.076609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:50.076609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:50.076609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:50.076609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:50.076609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:50.079318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 382/752] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:50.080231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed616c-1e5c-4897-b738-5eac086e0d1d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.165928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:50.175153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75622350-32e3-4f55-acb9-a449970540d2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:50.199746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed616c-1e5c-4897-b738-5eac086e0d1d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:50.212180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0a608-1fab-4acb-8d84-a477c358f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.213045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0a608-1fab-4acb-8d84-a477c358f1cb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.213574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0a608-1fab-4acb-8d84-a477c358f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.214007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0a608-1fab-4acb-8d84-a477c358f1cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.226392 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0a608-1fab-4acb-8d84-a477c358f1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.228159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0a608-1fab-4acb-8d84-a477c358f1cb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.240489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6636402-359a-4447-94e8-c98d89a29ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.241294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6636402-359a-4447-94e8-c98d89a29ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.241738 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6636402-359a-4447-94e8-c98d89a29ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.242115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6636402-359a-4447-94e8-c98d89a29ea1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-17b0a608-1fab-4acb-8d84-a477c358f1cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:50.245738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:50.247535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:50.248615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:50.248615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:50.248615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:50.248615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:50.250626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 371/753] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:50.251602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0a608-1fab-4acb-8d84-a477c358f1cb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.256948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6636402-359a-4447-94e8-c98d89a29ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.258627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6636402-359a-4447-94e8-c98d89a29ea1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c6636402-359a-4447-94e8-c98d89a29ea1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:50.272103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:50.273117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:50.274064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 383/754] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:50.275072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6636402-359a-4447-94e8-c98d89a29ea1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.339181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:50.360317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0a608-1fab-4acb-8d84-a477c358f1cb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:50.384699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6636402-359a-4447-94e8-c98d89a29ea1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:50.397865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddce66-42c4-47b9-9c24-e8563b5e31ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.398576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddce66-42c4-47b9-9c24-e8563b5e31ed None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.398854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddce66-42c4-47b9-9c24-e8563b5e31ed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.399136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddce66-42c4-47b9-9c24-e8563b5e31ed None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.399869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddce66-42c4-47b9-9c24-e8563b5e31ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:50.400168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddce66-42c4-47b9-9c24-e8563b5e31ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.401868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ddce66-42c4-47b9-9c24-e8563b5e31ed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.414506 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 372/755] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:50.415668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddce66-42c4-47b9-9c24-e8563b5e31ed None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.428240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:50.429475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74933fdb-1999-4cfa-850c-f783426d3c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.430366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74933fdb-1999-4cfa-850c-f783426d3c9e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.431029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74933fdb-1999-4cfa-850c-f783426d3c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.431500 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74933fdb-1999-4cfa-850c-f783426d3c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.434392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74933fdb-1999-4cfa-850c-f783426d3c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:50.434392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74933fdb-1999-4cfa-850c-f783426d3c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.434392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74933fdb-1999-4cfa-850c-f783426d3c9e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.452325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 384/756] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:50.453459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74933fdb-1999-4cfa-850c-f783426d3c9e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.525165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddce66-42c4-47b9-9c24-e8563b5e31ed None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:50.561219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74933fdb-1999-4cfa-850c-f783426d3c9e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:50.561760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba693ba-0413-4361-9be5-c81869788234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.562427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba693ba-0413-4361-9be5-c81869788234 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.562729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba693ba-0413-4361-9be5-c81869788234 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.562998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba693ba-0413-4361-9be5-c81869788234 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.563788 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba693ba-0413-4361-9be5-c81869788234 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:50.564088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba693ba-0413-4361-9be5-c81869788234 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.566095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba693ba-0413-4361-9be5-c81869788234 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.584916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 373/757] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:50.585768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba693ba-0413-4361-9be5-c81869788234 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.587052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:50.605186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed53a48-dbb0-4f74-bc5a-2f6c438c079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.606259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed53a48-dbb0-4f74-bc5a-2f6c438c079e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.606792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed53a48-dbb0-4f74-bc5a-2f6c438c079e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.607275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed53a48-dbb0-4f74-bc5a-2f6c438c079e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.608301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed53a48-dbb0-4f74-bc5a-2f6c438c079e None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=54bda2af62ee4ee39a9a350b7ce6430e) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:50.608853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed53a48-dbb0-4f74-bc5a-2f6c438c079e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.610601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed53a48-dbb0-4f74-bc5a-2f6c438c079e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.617368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 385/758] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/role_assignments?user.id=54bda2af62ee4ee39a9a350b7ce6430e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:50.618658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed53a48-dbb0-4f74-bc5a-2f6c438c079e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.720320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:50.799044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba693ba-0413-4361-9be5-c81869788234 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:50.831477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed53a48-dbb0-4f74-bc5a-2f6c438c079e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:50.837704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3d6a6-3bd4-4fb3-8224-1935c2f352de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.838352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3d6a6-3bd4-4fb3-8224-1935c2f352de None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.838650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3d6a6-3bd4-4fb3-8224-1935c2f352de None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.838915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3d6a6-3bd4-4fb3-8224-1935c2f352de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.846901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3d6a6-3bd4-4fb3-8224-1935c2f352de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.848407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc3d6a6-3bd4-4fb3-8224-1935c2f352de None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-dcc3d6a6-3bd4-4fb3-8224-1935c2f352de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:50.861383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:50.862304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:50.863919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:50.863919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:50.863919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:50.863919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:50.863919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:50.863919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:50.863919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:50.863919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:50.863919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 374/759] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:50.868711 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3d6a6-3bd4-4fb3-8224-1935c2f352de None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.876337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58d5b9-bf06-492f-8a09-fd4f451b209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:50.877339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58d5b9-bf06-492f-8a09-fd4f451b209a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:50.877823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58d5b9-bf06-492f-8a09-fd4f451b209a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:50.878296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58d5b9-bf06-492f-8a09-fd4f451b209a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:50.889876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58d5b9-bf06-492f-8a09-fd4f451b209a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:50.891599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58d5b9-bf06-492f-8a09-fd4f451b209a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ab58d5b9-bf06-492f-8a09-fd4f451b209a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:50.907695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:50.908642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:50.909534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:50.909534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:50.909534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:50.909534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:50.916052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 386/760] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:50.917554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58d5b9-bf06-492f-8a09-fd4f451b209a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:50.956841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:50.977925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc3d6a6-3bd4-4fb3-8224-1935c2f352de None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:51.005779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:51.016052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c3fa2-a149-4364-8bd7-6756dd8c904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:51.016512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c3fa2-a149-4364-8bd7-6756dd8c904e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.016790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c3fa2-a149-4364-8bd7-6756dd8c904e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.020257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c3fa2-a149-4364-8bd7-6756dd8c904e None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.020257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c3fa2-a149-4364-8bd7-6756dd8c904e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:51.020257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c3fa2-a149-4364-8bd7-6756dd8c904e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:51.020257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c3fa2-a149-4364-8bd7-6756dd8c904e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:51.035672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 375/761] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:51.041334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58d5b9-bf06-492f-8a09-fd4f451b209a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:51.041798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c3fa2-a149-4364-8bd7-6756dd8c904e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:51.079026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b6cf2-ba56-46c8-89fb-a1ee61a9a0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:51.080056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b6cf2-ba56-46c8-89fb-a1ee61a9a0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.080551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b6cf2-ba56-46c8-89fb-a1ee61a9a0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.081026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b6cf2-ba56-46c8-89fb-a1ee61a9a0c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.081956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b6cf2-ba56-46c8-89fb-a1ee61a9a0c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:51.082500 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b6cf2-ba56-46c8-89fb-a1ee61a9a0c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:51.084092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b6cf2-ba56-46c8-89fb-a1ee61a9a0c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:51.101614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 387/762] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:51.104224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b6cf2-ba56-46c8-89fb-a1ee61a9a0c4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:51.148745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c3fa2-a149-4364-8bd7-6756dd8c904e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:51.185136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaff36f-0645-4601-8a21-410710abe86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:51.185752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaff36f-0645-4601-8a21-410710abe86c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.186026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaff36f-0645-4601-8a21-410710abe86c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.186297 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaff36f-0645-4601-8a21-410710abe86c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.187106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaff36f-0645-4601-8a21-410710abe86c None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=e0f2d5364cb84337b77956dc5da78b4d) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:51.187410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaff36f-0645-4601-8a21-410710abe86c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:51.188823 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaff36f-0645-4601-8a21-410710abe86c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:51.194790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 376/763] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:51 2022] GET /identity/v3/role_assignments?user.id=e0f2d5364cb84337b77956dc5da78b4d&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:51.213772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:51.221365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b6cf2-ba56-46c8-89fb-a1ee61a9a0c4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:51.265535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb7637-0cfb-4c3b-bea1-1c38bde83ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:51.266640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb7637-0cfb-4c3b-bea1-1c38bde83ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.267191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb7637-0cfb-4c3b-bea1-1c38bde83ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.267669 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb7637-0cfb-4c3b-bea1-1c38bde83ae3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.276584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb7637-0cfb-4c3b-bea1-1c38bde83ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:51.278672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb7637-0cfb-4c3b-bea1-1c38bde83ae3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-54bb7637-0cfb-4c3b-bea1-1c38bde83ae3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:51.291781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:51.292736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:51.294231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:51.294231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:51.294231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:51.294231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:51.294231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:51.294231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:51.294231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:51.294231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:51.301372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 388/764] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:51.303210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaff36f-0645-4601-8a21-410710abe86c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:51.354430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332417c-c47b-401f-80a4-d6fbbb424a36 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.355085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332417c-c47b-401f-80a4-d6fbbb424a36 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.355565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332417c-c47b-401f-80a4-d6fbbb424a36 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.358145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 389/765] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:45:51.363404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c65d8-ea04-4c97-891c-d372f5439673 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.363978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c65d8-ea04-4c97-891c-d372f5439673 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.364455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c65d8-ea04-4c97-891c-d372f5439673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.384492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:51.402725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ff6c65d8-ea04-4c97-891c-d372f5439673 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:51.404948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff6c65d8-ea04-4c97-891c-d372f5439673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:51.440386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaff36f-0645-4601-8a21-410710abe86c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:51.479302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ba1d2-4a7c-4fa1-907b-1da632a7201c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:51.480279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ba1d2-4a7c-4fa1-907b-1da632a7201c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.480761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ba1d2-4a7c-4fa1-907b-1da632a7201c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.481987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ba1d2-4a7c-4fa1-907b-1da632a7201c None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.483055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ba1d2-4a7c-4fa1-907b-1da632a7201c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:51.483648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ba1d2-4a7c-4fa1-907b-1da632a7201c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:51.485348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ba1d2-4a7c-4fa1-907b-1da632a7201c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:51.504625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 377/766] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:51.507155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ba1d2-4a7c-4fa1-907b-1da632a7201c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:51.512147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 390/767] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 30 04:45:51.546720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c65d8-ea04-4c97-891c-d372f5439673 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:51.615218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ba1d2-4a7c-4fa1-907b-1da632a7201c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:51.629366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:51.653893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dc0e7-1012-4919-b0cd-7476923b7a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:51.655321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dc0e7-1012-4919-b0cd-7476923b7a55 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.655756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dc0e7-1012-4919-b0cd-7476923b7a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.656442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dc0e7-1012-4919-b0cd-7476923b7a55 None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/a1dc3ad467d9496cb3e0086a0cd8215e/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.658953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dc0e7-1012-4919-b0cd-7476923b7a55 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=a1dc3ad467d9496cb3e0086a0cd8215e, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:51.660947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dc0e7-1012-4919-b0cd-7476923b7a55 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:51.690677 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 378/768] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:45:51 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/a1dc3ad467d9496cb3e0086a0cd8215e/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:51.701353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:51.718713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c65d8-ea04-4c97-891c-d372f5439673 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:51.742263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efe5f1-d378-492e-af53-6c9bac3c853c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.742757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efe5f1-d378-492e-af53-6c9bac3c853c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.743136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efe5f1-d378-492e-af53-6c9bac3c853c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.750772 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 379/769] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:45:51.753054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c4661-6b4d-4948-a0d6-9a2a56a8ac4f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.753176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c4661-6b4d-4948-a0d6-9a2a56a8ac4f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.753549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c4661-6b4d-4948-a0d6-9a2a56a8ac4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.761851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1b5c0-ec72-4a65-8be3-8c7fa3fc0271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:51.763401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1b5c0-ec72-4a65-8be3-8c7fa3fc0271 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.764262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1b5c0-ec72-4a65-8be3-8c7fa3fc0271 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.764682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1b5c0-ec72-4a65-8be3-8c7fa3fc0271 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.765571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1b5c0-ec72-4a65-8be3-8c7fa3fc0271 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:51.767288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1b5c0-ec72-4a65-8be3-8c7fa3fc0271 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1fe1b5c0-ec72-4a65-8be3-8c7fa3fc0271 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:51.777955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:51.779176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 30 04:45:51.780251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:51.782926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 391/770] 10.222.0.20 () {56 vars in 1129 bytes} [Sat Jul 30 04:45:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:51.789036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1b5c0-ec72-4a65-8be3-8c7fa3fc0271 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:51.799719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fe6c4661-6b4d-4948-a0d6-9a2a56a8ac4f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:51.800981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fe6c4661-6b4d-4948-a0d6-9a2a56a8ac4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:51.928965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1b5c0-ec72-4a65-8be3-8c7fa3fc0271 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:51.935086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 380/771] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:45:51.957814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c4661-6b4d-4948-a0d6-9a2a56a8ac4f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:51.970661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfa5bc-eae1-4dc8-998c-6b570af3338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:51.971293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfa5bc-eae1-4dc8-998c-6b570af3338f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:51.971579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfa5bc-eae1-4dc8-998c-6b570af3338f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:51.971882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfa5bc-eae1-4dc8-998c-6b570af3338f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:51.972575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfa5bc-eae1-4dc8-998c-6b570af3338f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:51.972880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfa5bc-eae1-4dc8-998c-6b570af3338f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:51.974412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfa5bc-eae1-4dc8-998c-6b570af3338f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:51.980097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 392/772] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Jul 30 04:45:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:51.990799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfa5bc-eae1-4dc8-998c-6b570af3338f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:52.023172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:52.087914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c4661-6b4d-4948-a0d6-9a2a56a8ac4f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:52.089430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:52.129131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2f2b5-8ced-4be4-8481-eb09c282e5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:52.129783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2f2b5-8ced-4be4-8481-eb09c282e5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:52.130059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2f2b5-8ced-4be4-8481-eb09c282e5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:52.130324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2f2b5-8ced-4be4-8481-eb09c282e5e0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:52.131022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2f2b5-8ced-4be4-8481-eb09c282e5e0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:52.132333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2f2b5-8ced-4be4-8481-eb09c282e5e0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d1e2f2b5-8ced-4be4-8481-eb09c282e5e0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:52.143961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:52.145061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:52.145928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 381/773] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:51 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:52.148429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfa5bc-eae1-4dc8-998c-6b570af3338f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:52.151449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2f2b5-8ced-4be4-8481-eb09c282e5e0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:52.192595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc350b25-6196-4f28-a122-0ab46f760f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:52.193561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc350b25-6196-4f28-a122-0ab46f760f04 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:52.194070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc350b25-6196-4f28-a122-0ab46f760f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:52.194585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc350b25-6196-4f28-a122-0ab46f760f04 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:52.195855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc350b25-6196-4f28-a122-0ab46f760f04 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:52.196520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc350b25-6196-4f28-a122-0ab46f760f04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:52.198552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc350b25-6196-4f28-a122-0ab46f760f04 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:52.204877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 393/774] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Jul 30 04:45:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:52.210752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc350b25-6196-4f28-a122-0ab46f760f04 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:52.303990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:52.324319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2f2b5-8ced-4be4-8481-eb09c282e5e0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:52.329762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc350b25-6196-4f28-a122-0ab46f760f04 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:52.359559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efafb9-2804-465c-941a-0e1560fbcde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:52.360222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efafb9-2804-465c-941a-0e1560fbcde4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:52.360459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efafb9-2804-465c-941a-0e1560fbcde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:52.360731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efafb9-2804-465c-941a-0e1560fbcde4 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:52.361411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efafb9-2804-465c-941a-0e1560fbcde4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:52.361708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efafb9-2804-465c-941a-0e1560fbcde4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:52.363152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efafb9-2804-465c-941a-0e1560fbcde4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:52.367270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb28d81-74bf-4977-9502-de436ecc31c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:52.368147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb28d81-74bf-4977-9502-de436ecc31c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:52.368258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 382/775] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:45:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:52.368745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb28d81-74bf-4977-9502-de436ecc31c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:52.369221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb28d81-74bf-4977-9502-de436ecc31c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:52.370193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb28d81-74bf-4977-9502-de436ecc31c5 None admin] RBAC: Adding query filter params (interface=public, service_id=3a2cfdb3f71b427ea6f09e9ba84eccea, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:52.370736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb28d81-74bf-4977-9502-de436ecc31c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:52.372941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb28d81-74bf-4977-9502-de436ecc31c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:52.373778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efafb9-2804-465c-941a-0e1560fbcde4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:52.382038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 394/776] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Jul 30 04:45:52 2022] GET /identity/v3/endpoints?service_id=3a2cfdb3f71b427ea6f09e9ba84eccea&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:52.388433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb28d81-74bf-4977-9502-de436ecc31c5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:52.407712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:52.479996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efafb9-2804-465c-941a-0e1560fbcde4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:52.504559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb28d81-74bf-4977-9502-de436ecc31c5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:52.516514 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd479c-55fd-4bc6-a913-659ffb8daf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:52.516947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd479c-55fd-4bc6-a913-659ffb8daf1d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:52.517231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd479c-55fd-4bc6-a913-659ffb8daf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:52.517502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd479c-55fd-4bc6-a913-659ffb8daf1d None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:52.518242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd479c-55fd-4bc6-a913-659ffb8daf1d None admin] RBAC: Adding query filter params (type=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:52.518497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd479c-55fd-4bc6-a913-659ffb8daf1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:52.519872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd479c-55fd-4bc6-a913-659ffb8daf1d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:52.525374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 383/777] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:45:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:52.535752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd479c-55fd-4bc6-a913-659ffb8daf1d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:52.545222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c156969-ce2d-48fc-af14-27e367888e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:52.546154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c156969-ce2d-48fc-af14-27e367888e5f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:52.546702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c156969-ce2d-48fc-af14-27e367888e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:52.547187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c156969-ce2d-48fc-af14-27e367888e5f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:52.548065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c156969-ce2d-48fc-af14-27e367888e5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:52.549647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c156969-ce2d-48fc-af14-27e367888e5f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:52.555912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 395/778] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Jul 30 04:45:52 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:52.691003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:52.701537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd479c-55fd-4bc6-a913-659ffb8daf1d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:52.766469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ffc76-343a-4945-bb2f-b9347275982f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:52.767106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ffc76-343a-4945-bb2f-b9347275982f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:52.767388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ffc76-343a-4945-bb2f-b9347275982f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:52.767662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ffc76-343a-4945-bb2f-b9347275982f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:52.768444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ffc76-343a-4945-bb2f-b9347275982f None admin] RBAC: Adding query filter params (interface=public, service_id=fcbdef45a0cf422ca474666b50a2051e, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:52.768751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ffc76-343a-4945-bb2f-b9347275982f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:52.770187 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ffc76-343a-4945-bb2f-b9347275982f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:52.779807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 384/779] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Jul 30 04:45:52 2022] GET /identity/v3/endpoints?service_id=fcbdef45a0cf422ca474666b50a2051e&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:52.784832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c156969-ce2d-48fc-af14-27e367888e5f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:52.795115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:52.904404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c156969-ce2d-48fc-af14-27e367888e5f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:52.960228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fdfff-9327-4804-a1c1-ee3057a3318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:52.961220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fdfff-9327-4804-a1c1-ee3057a3318e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:52.961728 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fdfff-9327-4804-a1c1-ee3057a3318e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:52.962233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fdfff-9327-4804-a1c1-ee3057a3318e None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:52.963111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fdfff-9327-4804-a1c1-ee3057a3318e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:52.964683 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fdfff-9327-4804-a1c1-ee3057a3318e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:52.971414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 396/780] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Jul 30 04:45:52 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:53.098153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b9385-ff3f-40a9-bfa2-09418c5e4d2a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:53.098702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b9385-ff3f-40a9-bfa2-09418c5e4d2a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:53.098702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b9385-ff3f-40a9-bfa2-09418c5e4d2a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:53.100468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 385/781] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:53.106810 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082f309-3671-4007-8c75-0138d990fff7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:53.107488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082f309-3671-4007-8c75-0138d990fff7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:53.107999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082f309-3671-4007-8c75-0138d990fff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:53.135658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5082f309-3671-4007-8c75-0138d990fff7 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:53.137299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5082f309-3671-4007-8c75-0138d990fff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:53.221843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:53.227838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 397/782] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:45:53.251277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b9385-ff3f-40a9-bfa2-09418c5e4d2a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:53.391837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b9385-ff3f-40a9-bfa2-09418c5e4d2a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:53.440800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415ad15-b915-4df6-af3a-bd71f4b27402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:53.441860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415ad15-b915-4df6-af3a-bd71f4b27402 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:53.442353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415ad15-b915-4df6-af3a-bd71f4b27402 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:53.442881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415ad15-b915-4df6-af3a-bd71f4b27402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:53.467460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415ad15-b915-4df6-af3a-bd71f4b27402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5415ad15-b915-4df6-af3a-bd71f4b27402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:53.473348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:53.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:53.475166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:53.476302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 386/783] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:53.481717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082f309-3671-4007-8c75-0138d990fff7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:53.647311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082f309-3671-4007-8c75-0138d990fff7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:53.699832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f217d-8199-40a4-bf49-0257a904e3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:53.700531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f217d-8199-40a4-bf49-0257a904e3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:53.700757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f217d-8199-40a4-bf49-0257a904e3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:53.701022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f217d-8199-40a4-bf49-0257a904e3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:53.701753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f217d-8199-40a4-bf49-0257a904e3a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:53.702057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f217d-8199-40a4-bf49-0257a904e3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:53.703468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f217d-8199-40a4-bf49-0257a904e3a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:53.714357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 398/784] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:53.720270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415ad15-b915-4df6-af3a-bd71f4b27402 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:53.757961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:53.869001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415ad15-b915-4df6-af3a-bd71f4b27402 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:53.921399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59e75f-bc34-4b2d-890f-8bf5049d4932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:53.921973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59e75f-bc34-4b2d-890f-8bf5049d4932 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:53.922251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59e75f-bc34-4b2d-890f-8bf5049d4932 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:53.922547 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59e75f-bc34-4b2d-890f-8bf5049d4932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:53.939366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59e75f-bc34-4b2d-890f-8bf5049d4932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:53.940810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59e75f-bc34-4b2d-890f-8bf5049d4932 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:53.948981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d15c65-9ef4-4c7b-917e-e5a402f1bfe8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:53.949541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d15c65-9ef4-4c7b-917e-e5a402f1bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:53.950605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d15c65-9ef4-4c7b-917e-e5a402f1bfe8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:53.956173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 399/785] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ca59e75f-bc34-4b2d-890f-8bf5049d4932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:53.959445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:53.960320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:53.961202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:53.961202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:53.961202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:53.961202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:53.961202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 387/786] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:53.963282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea8506-4fbd-4edc-b863-5e882acf8ca7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:53.963742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea8506-4fbd-4edc-b863-5e882acf8ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:53.964175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea8506-4fbd-4edc-b863-5e882acf8ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:53.966619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59e75f-bc34-4b2d-890f-8bf5049d4932 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:53.984531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ebea8506-4fbd-4edc-b863-5e882acf8ca7 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:53.986036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ebea8506-4fbd-4edc-b863-5e882acf8ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:54.062742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:54.096522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59e75f-bc34-4b2d-890f-8bf5049d4932 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:54.106254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:54.133894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 400/787] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:45:54.150015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce8fc-a5d4-41cf-add4-df37460848ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:54.151526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce8fc-a5d4-41cf-add4-df37460848ba None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.151954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce8fc-a5d4-41cf-add4-df37460848ba None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.152371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce8fc-a5d4-41cf-add4-df37460848ba None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.153306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce8fc-a5d4-41cf-add4-df37460848ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:54.153754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce8fc-a5d4-41cf-add4-df37460848ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:54.155321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce8fc-a5d4-41cf-add4-df37460848ba None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:54.180249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 388/788] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:54.185747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea8506-4fbd-4edc-b863-5e882acf8ca7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:54.189071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce8fc-a5d4-41cf-add4-df37460848ba None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:54.329529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce8fc-a5d4-41cf-add4-df37460848ba None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:54.342957 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea8506-4fbd-4edc-b863-5e882acf8ca7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:54.361865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:54.369620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915221b-448f-4b39-8d97-8d88ff8428c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:54.370389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915221b-448f-4b39-8d97-8d88ff8428c3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.370687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915221b-448f-4b39-8d97-8d88ff8428c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.370953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915221b-448f-4b39-8d97-8d88ff8428c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.377881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821364c-60c3-4a45-a120-64eb970c650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:54.378552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821364c-60c3-4a45-a120-64eb970c650c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.378836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821364c-60c3-4a45-a120-64eb970c650c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.379111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821364c-60c3-4a45-a120-64eb970c650c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.383693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915221b-448f-4b39-8d97-8d88ff8428c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:54.386540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821364c-60c3-4a45-a120-64eb970c650c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:54.388581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821364c-60c3-4a45-a120-64eb970c650c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-9915221b-448f-4b39-8d97-8d88ff8428c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:54.389895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:54.390851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:45:54.391705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:54.392865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 389/789] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:45:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:54.393961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3406d-2434-4adc-94d5-58a2e56b8a2e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.393961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3406d-2434-4adc-94d5-58a2e56b8a2e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.394134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3406d-2434-4adc-94d5-58a2e56b8a2e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.395867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 390/790] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:54.399383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3406d-2434-4adc-94d5-58a2e56b8a2e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a821364c-60c3-4a45-a120-64eb970c650c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:54.405033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:54.406271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 401/791] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:54.407848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda48e9-eb79-4d2c-9da6-e00bc08c7270 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.409202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda48e9-eb79-4d2c-9da6-e00bc08c7270 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.409202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda48e9-eb79-4d2c-9da6-e00bc08c7270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.428626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8dda48e9-eb79-4d2c-9da6-e00bc08c7270 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:54.429836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8dda48e9-eb79-4d2c-9da6-e00bc08c7270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:54.482920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:54.510531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 402/792] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:45:54.511411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda48e9-eb79-4d2c-9da6-e00bc08c7270 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:54.544862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3406d-2434-4adc-94d5-58a2e56b8a2e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:54.592654 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36756396-a308-4edd-9183-5cd6055ce038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:54.593440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36756396-a308-4edd-9183-5cd6055ce038 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.593821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36756396-a308-4edd-9183-5cd6055ce038 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.594197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36756396-a308-4edd-9183-5cd6055ce038 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.595072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36756396-a308-4edd-9183-5cd6055ce038 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:54.595481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36756396-a308-4edd-9183-5cd6055ce038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:54.596956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36756396-a308-4edd-9183-5cd6055ce038 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:54.606916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 391/793] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:45:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:54.607961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36756396-a308-4edd-9183-5cd6055ce038 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:54.642831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda48e9-eb79-4d2c-9da6-e00bc08c7270 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:54.663352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:54.684889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb782-fa33-4968-b144-598d732e3036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:54.685766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb782-fa33-4968-b144-598d732e3036 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.686189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb782-fa33-4968-b144-598d732e3036 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.686639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb782-fa33-4968-b144-598d732e3036 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.687657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb782-fa33-4968-b144-598d732e3036 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:54.688122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb782-fa33-4968-b144-598d732e3036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:54.690054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb782-fa33-4968-b144-598d732e3036 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:54.711278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 403/794] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Jul 30 04:45:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:54.712494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb782-fa33-4968-b144-598d732e3036 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:54.713415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36756396-a308-4edd-9183-5cd6055ce038 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:54.750836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413718-b699-41e7-abc7-9da7d9d47dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:54.751548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413718-b699-41e7-abc7-9da7d9d47dda None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.751927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413718-b699-41e7-abc7-9da7d9d47dda None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.752331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413718-b699-41e7-abc7-9da7d9d47dda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.761802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:54.764163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06413718-b699-41e7-abc7-9da7d9d47dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-06413718-b699-41e7-abc7-9da7d9d47dda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:54.769413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:54.770758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:54.771622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:54.776592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 392/795] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:54.777450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413718-b699-41e7-abc7-9da7d9d47dda None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:54.832027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb782-fa33-4968-b144-598d732e3036 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:54.869545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49768fd7-f504-4b2f-b398-b52ce7c7e63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:54.870728 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49768fd7-f504-4b2f-b398-b52ce7c7e63b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.871241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49768fd7-f504-4b2f-b398-b52ce7c7e63b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.872341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49768fd7-f504-4b2f-b398-b52ce7c7e63b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.883802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49768fd7-f504-4b2f-b398-b52ce7c7e63b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:54.885733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49768fd7-f504-4b2f-b398-b52ce7c7e63b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:54.888968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413718-b699-41e7-abc7-9da7d9d47dda None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:54.903561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-49768fd7-f504-4b2f-b398-b52ce7c7e63b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:54.904220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:54.905122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:54.906019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:54.906019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:54.906019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:54.906019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:54.908934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 404/796] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:54.909943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49768fd7-f504-4b2f-b398-b52ce7c7e63b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:54.926837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c97f4-238f-4225-bc95-3a7951ed8b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:54.927480 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c97f4-238f-4225-bc95-3a7951ed8b45 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:54.927841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c97f4-238f-4225-bc95-3a7951ed8b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:54.928028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c97f4-238f-4225-bc95-3a7951ed8b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:54.939547 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c97f4-238f-4225-bc95-3a7951ed8b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:54.941014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c97f4-238f-4225-bc95-3a7951ed8b45 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a83c97f4-238f-4225-bc95-3a7951ed8b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:54.957418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:54.958361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:54.959439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:54.959439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:54.959439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:54.959439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:54.959439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 393/797] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:54.959802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c97f4-238f-4225-bc95-3a7951ed8b45 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.020130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:55.030934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49768fd7-f504-4b2f-b398-b52ce7c7e63b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.069952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba945a-1f34-4817-95c5-582ea6abdbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.070715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba945a-1f34-4817-95c5-582ea6abdbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.071265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba945a-1f34-4817-95c5-582ea6abdbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.071477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba945a-1f34-4817-95c5-582ea6abdbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.072263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba945a-1f34-4817-95c5-582ea6abdbe5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:55.072626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba945a-1f34-4817-95c5-582ea6abdbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.074102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba945a-1f34-4817-95c5-582ea6abdbe5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.075065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c97f4-238f-4225-bc95-3a7951ed8b45 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.088320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 405/798] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:55.088320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba945a-1f34-4817-95c5-582ea6abdbe5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.123106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4205ea8-aac3-4f14-b8f7-76bc158a2953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.124179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4205ea8-aac3-4f14-b8f7-76bc158a2953 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.124832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4205ea8-aac3-4f14-b8f7-76bc158a2953 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.125108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4205ea8-aac3-4f14-b8f7-76bc158a2953 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.126072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4205ea8-aac3-4f14-b8f7-76bc158a2953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:55.126382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4205ea8-aac3-4f14-b8f7-76bc158a2953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.128038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4205ea8-aac3-4f14-b8f7-76bc158a2953 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.144844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 394/799] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:55.145740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4205ea8-aac3-4f14-b8f7-76bc158a2953 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.173466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:55.200842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba945a-1f34-4817-95c5-582ea6abdbe5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.240933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e367fc6-2868-4098-b223-055da7fb4db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.241550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e367fc6-2868-4098-b223-055da7fb4db6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.241820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e367fc6-2868-4098-b223-055da7fb4db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.242093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e367fc6-2868-4098-b223-055da7fb4db6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.242819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e367fc6-2868-4098-b223-055da7fb4db6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:55.243116 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e367fc6-2868-4098-b223-055da7fb4db6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.244471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e367fc6-2868-4098-b223-055da7fb4db6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.256466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4205ea8-aac3-4f14-b8f7-76bc158a2953 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.257091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 406/800] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:55.258306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c6cac-af3f-4587-bf02-fd98232cb158 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.258618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c6cac-af3f-4587-bf02-fd98232cb158 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.258888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c6cac-af3f-4587-bf02-fd98232cb158 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.262487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 407/801] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:55.262487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c6cac-af3f-4587-bf02-fd98232cb158 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.277290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:55.304091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c251d-eba8-4e38-8f3b-db53c4159e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.304845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c251d-eba8-4e38-8f3b-db53c4159e85 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.305179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c251d-eba8-4e38-8f3b-db53c4159e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.305453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c251d-eba8-4e38-8f3b-db53c4159e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.316786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c251d-eba8-4e38-8f3b-db53c4159e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.318224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c251d-eba8-4e38-8f3b-db53c4159e85 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a15c251d-eba8-4e38-8f3b-db53c4159e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:55.334163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:55.335107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:55.336019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:55.336019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:55.336019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:55.336019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:55.336019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 395/802] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:55.336312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c251d-eba8-4e38-8f3b-db53c4159e85 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.369389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c6cac-af3f-4587-bf02-fd98232cb158 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.411465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51ed55-a416-48f6-9b4f-40e5fcff6bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.412115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51ed55-a416-48f6-9b4f-40e5fcff6bad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.412398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51ed55-a416-48f6-9b4f-40e5fcff6bad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.412679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51ed55-a416-48f6-9b4f-40e5fcff6bad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.420660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51ed55-a416-48f6-9b4f-40e5fcff6bad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.421467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:55.422279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51ed55-a416-48f6-9b4f-40e5fcff6bad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ca51ed55-a416-48f6-9b4f-40e5fcff6bad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:55.435265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:55.436439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:55.437387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 408/803] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:55.437972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5b149-6c68-49fd-9bfe-7fa41e9efb86 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.437972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5b149-6c68-49fd-9bfe-7fa41e9efb86 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.438163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5b149-6c68-49fd-9bfe-7fa41e9efb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.453544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c251d-eba8-4e38-8f3b-db53c4159e85 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.456543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e0c5b149-6c68-49fd-9bfe-7fa41e9efb86 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:55.457743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e0c5b149-6c68-49fd-9bfe-7fa41e9efb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:55.491986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10503bcc-31e2-428e-a4e3-4c7d1ae7d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.492601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10503bcc-31e2-428e-a4e3-4c7d1ae7d0fc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.492877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10503bcc-31e2-428e-a4e3-4c7d1ae7d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.493138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10503bcc-31e2-428e-a4e3-4c7d1ae7d0fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.500489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10503bcc-31e2-428e-a4e3-4c7d1ae7d0fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.501930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10503bcc-31e2-428e-a4e3-4c7d1ae7d0fc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-10503bcc-31e2-428e-a4e3-4c7d1ae7d0fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:55.514127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:55.515257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 396/804] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:55.516152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10503bcc-31e2-428e-a4e3-4c7d1ae7d0fc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.532056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 409/805] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:45:55.532907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5b149-6c68-49fd-9bfe-7fa41e9efb86 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.540214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:55.618540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10503bcc-31e2-428e-a4e3-4c7d1ae7d0fc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.640998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5b149-6c68-49fd-9bfe-7fa41e9efb86 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.652023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb236e-270f-4fd6-ba48-cf285da08357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.652675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb236e-270f-4fd6-ba48-cf285da08357 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.652953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb236e-270f-4fd6-ba48-cf285da08357 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.653218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb236e-270f-4fd6-ba48-cf285da08357 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.653899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb236e-270f-4fd6-ba48-cf285da08357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:55.654196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb236e-270f-4fd6-ba48-cf285da08357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.655588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb236e-270f-4fd6-ba48-cf285da08357 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.657830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:55.667822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 397/806] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:55.669602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de7d51-17c0-43ef-aa76-8a4ff65d5cef None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.669875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de7d51-17c0-43ef-aa76-8a4ff65d5cef None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.670140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de7d51-17c0-43ef-aa76-8a4ff65d5cef None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.671361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 398/807] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:55.672040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de7d51-17c0-43ef-aa76-8a4ff65d5cef None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.681310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5341696-b175-41a3-a1a4-b659f2f7ffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.681932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5341696-b175-41a3-a1a4-b659f2f7ffeb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.682210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5341696-b175-41a3-a1a4-b659f2f7ffeb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.682508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5341696-b175-41a3-a1a4-b659f2f7ffeb None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.683307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5341696-b175-41a3-a1a4-b659f2f7ffeb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:55.683613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5341696-b175-41a3-a1a4-b659f2f7ffeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.685635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5341696-b175-41a3-a1a4-b659f2f7ffeb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.702807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 410/808] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:55.703613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5341696-b175-41a3-a1a4-b659f2f7ffeb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.765942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:55.778575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de7d51-17c0-43ef-aa76-8a4ff65d5cef None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.804748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5341696-b175-41a3-a1a4-b659f2f7ffeb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.813549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d77e67-dd76-4e71-a3a3-9804b25f57d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.814168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d77e67-dd76-4e71-a3a3-9804b25f57d2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.814460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d77e67-dd76-4e71-a3a3-9804b25f57d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.814749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d77e67-dd76-4e71-a3a3-9804b25f57d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.815498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d77e67-dd76-4e71-a3a3-9804b25f57d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:55.815800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d77e67-dd76-4e71-a3a3-9804b25f57d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.817228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d77e67-dd76-4e71-a3a3-9804b25f57d2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.828477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 399/809] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:55.829262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d77e67-dd76-4e71-a3a3-9804b25f57d2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.839313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23997093-0953-4932-87aa-27812d33470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.839960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997093-0953-4932-87aa-27812d33470e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.840243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997093-0953-4932-87aa-27812d33470e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.840519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997093-0953-4932-87aa-27812d33470e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.841326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23997093-0953-4932-87aa-27812d33470e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.842754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23997093-0953-4932-87aa-27812d33470e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-23997093-0953-4932-87aa-27812d33470e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:55.852436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:55.853530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:55.855250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 411/810] 10.222.0.20 () {56 vars in 1129 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:55.857027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd851-8cbd-4a65-bbf3-152714937a34 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.857027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd851-8cbd-4a65-bbf3-152714937a34 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.857027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd851-8cbd-4a65-bbf3-152714937a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.873926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-98bcd851-8cbd-4a65-bbf3-152714937a34 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:55.875143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98bcd851-8cbd-4a65-bbf3-152714937a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:55.890729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:55.935215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d77e67-dd76-4e71-a3a3-9804b25f57d2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:55.942208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 412/811] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:45:55.943204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcd851-8cbd-4a65-bbf3-152714937a34 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:55.977943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f45e-a80b-4d98-9851-46fea6c19b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:55.978623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f45e-a80b-4d98-9851-46fea6c19b8f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:55.978906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f45e-a80b-4d98-9851-46fea6c19b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:55.979182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f45e-a80b-4d98-9851-46fea6c19b8f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:55.986811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6f45e-a80b-4d98-9851-46fea6c19b8f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:55.988239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6f45e-a80b-4d98-9851-46fea6c19b8f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:55.998731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4cd6f45e-a80b-4d98-9851-46fea6c19b8f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:55.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:56.000822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:56.001827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 400/812] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:56.002413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f45e-a80b-4d98-9851-46fea6c19b8f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.048367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcd851-8cbd-4a65-bbf3-152714937a34 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.082921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6900-21b9-46ca-92a7-292a23434cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.083499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6900-21b9-46ca-92a7-292a23434cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.083771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6900-21b9-46ca-92a7-292a23434cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.084042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6900-21b9-46ca-92a7-292a23434cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.094206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e6900-21b9-46ca-92a7-292a23434cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.095676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e6900-21b9-46ca-92a7-292a23434cb7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.108045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f45e-a80b-4d98-9851-46fea6c19b8f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eb0e6900-21b9-46ca-92a7-292a23434cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:56.111360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:56.112443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:56.113700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:56.113700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:56.113700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:56.113700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:56.113700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 413/813] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:56.113700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6900-21b9-46ca-92a7-292a23434cb7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.125865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:56.144827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7bc33-c50d-4d15-b662-486bac8799b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.145497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7bc33-c50d-4d15-b662-486bac8799b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.145777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7bc33-c50d-4d15-b662-486bac8799b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.146055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7bc33-c50d-4d15-b662-486bac8799b8 None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/54bda2af62ee4ee39a9a350b7ce6430e/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.148138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7bc33-c50d-4d15-b662-486bac8799b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=54bda2af62ee4ee39a9a350b7ce6430e, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.149569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7bc33-c50d-4d15-b662-486bac8799b8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.163018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 401/814] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:45:56 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/54bda2af62ee4ee39a9a350b7ce6430e/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:56.163603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7bc33-c50d-4d15-b662-486bac8799b8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.236035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6900-21b9-46ca-92a7-292a23434cb7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.246956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:56.278324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059260b8-ea88-4d1f-b5da-91a75032c5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.279027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059260b8-ea88-4d1f-b5da-91a75032c5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.279308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059260b8-ea88-4d1f-b5da-91a75032c5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.279587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059260b8-ea88-4d1f-b5da-91a75032c5a3 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.280282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059260b8-ea88-4d1f-b5da-91a75032c5a3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:56.280640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059260b8-ea88-4d1f-b5da-91a75032c5a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.282044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059260b8-ea88-4d1f-b5da-91a75032c5a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.282882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7bc33-c50d-4d15-b662-486bac8799b8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.292380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 414/815] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Jul 30 04:45:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:56.292380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059260b8-ea88-4d1f-b5da-91a75032c5a3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.331423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74410227-8eea-421e-9a35-62c111621300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.332158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74410227-8eea-421e-9a35-62c111621300 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.332501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74410227-8eea-421e-9a35-62c111621300 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.332768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74410227-8eea-421e-9a35-62c111621300 None admin] PATH_INFO: `/v3/services/network` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.333451 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74410227-8eea-421e-9a35-62c111621300 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.335025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74410227-8eea-421e-9a35-62c111621300 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-74410227-8eea-421e-9a35-62c111621300 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:56.344982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:56.346199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:56.347510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 402/816] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:56 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:56.348327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74410227-8eea-421e-9a35-62c111621300 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.392283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:56.413424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059260b8-ea88-4d1f-b5da-91a75032c5a3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.451353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a0c8b-151e-4f48-977c-9c1df8909e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.451971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a0c8b-151e-4f48-977c-9c1df8909e22 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.452250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a0c8b-151e-4f48-977c-9c1df8909e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.452522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a0c8b-151e-4f48-977c-9c1df8909e22 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.453447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a0c8b-151e-4f48-977c-9c1df8909e22 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:56.453828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a0c8b-151e-4f48-977c-9c1df8909e22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.455352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a0c8b-151e-4f48-977c-9c1df8909e22 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.456619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74410227-8eea-421e-9a35-62c111621300 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.473853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 415/817] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Jul 30 04:45:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:56.474775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a0c8b-151e-4f48-977c-9c1df8909e22 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.496721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48461a-a8e4-46dd-b3c7-3c567c7930d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.497497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48461a-a8e4-46dd-b3c7-3c567c7930d3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.497877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48461a-a8e4-46dd-b3c7-3c567c7930d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.498248 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48461a-a8e4-46dd-b3c7-3c567c7930d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.499073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48461a-a8e4-46dd-b3c7-3c567c7930d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:56.499473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48461a-a8e4-46dd-b3c7-3c567c7930d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.500938 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48461a-a8e4-46dd-b3c7-3c567c7930d3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.507734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:56.514106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 403/818] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:56.515086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48461a-a8e4-46dd-b3c7-3c567c7930d3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.591257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a0c8b-151e-4f48-977c-9c1df8909e22 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.628139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:56.629176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d82db8-3bb5-4963-8758-209c2e7cdcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.629646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d82db8-3bb5-4963-8758-209c2e7cdcfc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.629986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d82db8-3bb5-4963-8758-209c2e7cdcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.630279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d82db8-3bb5-4963-8758-209c2e7cdcfc None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.630839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48461a-a8e4-46dd-b3c7-3c567c7930d3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.630956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d82db8-3bb5-4963-8758-209c2e7cdcfc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:56.631264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d82db8-3bb5-4963-8758-209c2e7cdcfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.632624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d82db8-3bb5-4963-8758-209c2e7cdcfc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.642820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 416/819] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Jul 30 04:45:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:56.642820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d82db8-3bb5-4963-8758-209c2e7cdcfc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.674416 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619d2b-cf00-47b9-b86d-313c138a0acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.675320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619d2b-cf00-47b9-b86d-313c138a0acd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.675978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619d2b-cf00-47b9-b86d-313c138a0acd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.676345 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619d2b-cf00-47b9-b86d-313c138a0acd None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.677237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619d2b-cf00-47b9-b86d-313c138a0acd None admin] RBAC: Adding query filter params (name=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:56.677790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619d2b-cf00-47b9-b86d-313c138a0acd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.679263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619d2b-cf00-47b9-b86d-313c138a0acd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.689087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 404/820] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:45:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:56.689087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619d2b-cf00-47b9-b86d-313c138a0acd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.747699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:56.754797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d82db8-3bb5-4963-8758-209c2e7cdcfc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.790408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e2911-77a7-4bbd-9bc7-ce085afca96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.791154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e2911-77a7-4bbd-9bc7-ce085afca96e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.791506 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e2911-77a7-4bbd-9bc7-ce085afca96e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.791859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e2911-77a7-4bbd-9bc7-ce085afca96e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.798850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619d2b-cf00-47b9-b86d-313c138a0acd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.834374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadf8ce-75be-423c-a635-41117c074c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:56.835242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf8ce-75be-423c-a635-41117c074c7b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:56.835519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf8ce-75be-423c-a635-41117c074c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:56.835806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf8ce-75be-423c-a635-41117c074c7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:56.843691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadf8ce-75be-423c-a635-41117c074c7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.845175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadf8ce-75be-423c-a635-41117c074c7b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8aadf8ce-75be-423c-a635-41117c074c7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:56.857259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:56.858426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 405/821] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:56.859385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadf8ce-75be-423c-a635-41117c074c7b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:56.866022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e2911-77a7-4bbd-9bc7-ce085afca96e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:56.867519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e2911-77a7-4bbd-9bc7-ce085afca96e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:56.885996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:56.967227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:56.971783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadf8ce-75be-423c-a635-41117c074c7b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:56.975622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 417/822] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:45:56.976587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e2911-77a7-4bbd-9bc7-ce085afca96e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:57.011814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1d5e4-a76a-462c-828e-84a5325f0efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:57.012626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1d5e4-a76a-462c-828e-84a5325f0efe None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:57.013004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1d5e4-a76a-462c-828e-84a5325f0efe None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:57.013371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1d5e4-a76a-462c-828e-84a5325f0efe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:57.014193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1d5e4-a76a-462c-828e-84a5325f0efe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:57.015697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1d5e4-a76a-462c-828e-84a5325f0efe None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:57.042837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 406/823] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:45:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:57.044557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1d5e4-a76a-462c-828e-84a5325f0efe None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:57.118040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e2911-77a7-4bbd-9bc7-ce085afca96e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:57.144751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:57.179180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1d5e4-a76a-462c-828e-84a5325f0efe None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:57.180453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef38cd-01b1-4dba-aca2-eb0174d5c634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:57.181385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef38cd-01b1-4dba-aca2-eb0174d5c634 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:57.181820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef38cd-01b1-4dba-aca2-eb0174d5c634 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:57.182348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef38cd-01b1-4dba-aca2-eb0174d5c634 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:57.183274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef38cd-01b1-4dba-aca2-eb0174d5c634 None admin] RBAC: Adding query filter params (type=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:57.183878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef38cd-01b1-4dba-aca2-eb0174d5c634 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:57.186005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef38cd-01b1-4dba-aca2-eb0174d5c634 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:57.192411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 418/824] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:45:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:57.193628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef38cd-01b1-4dba-aca2-eb0174d5c634 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:57.222648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edf1cc-1719-4393-93d5-ed8e5f45c5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:57.223608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edf1cc-1719-4393-93d5-ed8e5f45c5dd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:57.223886 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edf1cc-1719-4393-93d5-ed8e5f45c5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:57.224157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edf1cc-1719-4393-93d5-ed8e5f45c5dd None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:57.224913 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edf1cc-1719-4393-93d5-ed8e5f45c5dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:57.225202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edf1cc-1719-4393-93d5-ed8e5f45c5dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:57.226782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edf1cc-1719-4393-93d5-ed8e5f45c5dd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:57.240530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 407/825] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:57.241267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edf1cc-1719-4393-93d5-ed8e5f45c5dd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:57.258156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:57.320999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef38cd-01b1-4dba-aca2-eb0174d5c634 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:57.369471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4c3e8-8f34-4ec0-8093-a7931c075d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:57.370618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4c3e8-8f34-4ec0-8093-a7931c075d88 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:57.371045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4c3e8-8f34-4ec0-8093-a7931c075d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:57.371841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4c3e8-8f34-4ec0-8093-a7931c075d88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:57.378688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edf1cc-1719-4393-93d5-ed8e5f45c5dd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:57.385656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4c3e8-8f34-4ec0-8093-a7931c075d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:57.387856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4c3e8-8f34-4ec0-8093-a7931c075d88 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fcd4c3e8-8f34-4ec0-8093-a7931c075d88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:57.407236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:57.408170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:57.409057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:57.409057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:57.409057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:57.409057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:57.415651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 419/826] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:57.415651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4c3e8-8f34-4ec0-8093-a7931c075d88 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:57.427194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95141580-5699-4451-89cb-c37d105cff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:57.428175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95141580-5699-4451-89cb-c37d105cff77 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:57.428659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95141580-5699-4451-89cb-c37d105cff77 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:57.429133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95141580-5699-4451-89cb-c37d105cff77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:57.430040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95141580-5699-4451-89cb-c37d105cff77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:57.431637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95141580-5699-4451-89cb-c37d105cff77 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:57.438570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:57.462035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 408/827] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:45:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:45:57.463331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95141580-5699-4451-89cb-c37d105cff77 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:57.528968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4c3e8-8f34-4ec0-8093-a7931c075d88 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:57.536945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:57.597791 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ecaf-5445-4e73-aff6-63194a11d2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:57.598695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ecaf-5445-4e73-aff6-63194a11d2c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:57.599072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ecaf-5445-4e73-aff6-63194a11d2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:57.599441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ecaf-5445-4e73-aff6-63194a11d2c6 None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/e0f2d5364cb84337b77956dc5da78b4d/roles/4785312981e946918a58301ed5112c34` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:57.618750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684ecaf-5445-4e73-aff6-63194a11d2c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=e0f2d5364cb84337b77956dc5da78b4d, role_id=4785312981e946918a58301ed5112c34)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:57.620644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684ecaf-5445-4e73-aff6-63194a11d2c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:57.660551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 420/828] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:45:57 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/e0f2d5364cb84337b77956dc5da78b4d/roles/4785312981e946918a58301ed5112c34 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:45:57.662032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95141580-5699-4451-89cb-c37d105cff77 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:57.702682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39e792-b928-41cd-80ea-d5041f26318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:57.703347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39e792-b928-41cd-80ea-d5041f26318e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:57.703623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39e792-b928-41cd-80ea-d5041f26318e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:57.703890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39e792-b928-41cd-80ea-d5041f26318e None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:57.704595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39e792-b928-41cd-80ea-d5041f26318e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:57.704899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39e792-b928-41cd-80ea-d5041f26318e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:57.706291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39e792-b928-41cd-80ea-d5041f26318e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:57.721803 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 409/829] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:57.727797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ecaf-5445-4e73-aff6-63194a11d2c6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:57.868889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ecaf-5445-4e73-aff6-63194a11d2c6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:57.907134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7cd4b-57c3-4a06-a9db-4a8085ae4346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:57.907715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7cd4b-57c3-4a06-a9db-4a8085ae4346 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:57.908056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7cd4b-57c3-4a06-a9db-4a8085ae4346 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:57.908326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7cd4b-57c3-4a06-a9db-4a8085ae4346 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:57.915771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7cd4b-57c3-4a06-a9db-4a8085ae4346 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:57.917242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7cd4b-57c3-4a06-a9db-4a8085ae4346 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6cf7cd4b-57c3-4a06-a9db-4a8085ae4346 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:45:57.930212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:57.931199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:57.932097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:45:57.932097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:45:57.932097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:45:57.932097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:45:57.932097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:45:57.932097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:45:57.932097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:45:57.932097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:57.932097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 421/830] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:57.938759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39e792-b928-41cd-80ea-d5041f26318e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:57.994316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:58.047691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39e792-b928-41cd-80ea-d5041f26318e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:58.083883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea761f6-c544-4fc7-ac37-ebcd3e226d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:58.084888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea761f6-c544-4fc7-ac37-ebcd3e226d59 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:58.085382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea761f6-c544-4fc7-ac37-ebcd3e226d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:58.085853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea761f6-c544-4fc7-ac37-ebcd3e226d59 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:58.086829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea761f6-c544-4fc7-ac37-ebcd3e226d59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:58.087334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea761f6-c544-4fc7-ac37-ebcd3e226d59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:58.088978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea761f6-c544-4fc7-ac37-ebcd3e226d59 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:58.101980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 410/831] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:45:58.108182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7cd4b-57c3-4a06-a9db-4a8085ae4346 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:58.129454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:58.219330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7cd4b-57c3-4a06-a9db-4a8085ae4346 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:58.240027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3ebd8-618e-43aa-95e3-33e821909f7e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:58.240333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3ebd8-618e-43aa-95e3-33e821909f7e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:58.240578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3ebd8-618e-43aa-95e3-33e821909f7e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:58.241943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 411/832] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:58.248280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d48ed7-e45c-4a2e-bb01-fa93bf6679ec None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:58.248816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d48ed7-e45c-4a2e-bb01-fa93bf6679ec None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:58.249296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d48ed7-e45c-4a2e-bb01-fa93bf6679ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:58.263336 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7c1f-6542-440d-9d48-a623ee0b798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:58.263957 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7c1f-6542-440d-9d48-a623ee0b798a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:58.264253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7c1f-6542-440d-9d48-a623ee0b798a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:58.264902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7c1f-6542-440d-9d48-a623ee0b798a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:58.265386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7c1f-6542-440d-9d48-a623ee0b798a None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=a1dc3ad467d9496cb3e0086a0cd8215e) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:58.265695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7c1f-6542-440d-9d48-a623ee0b798a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:58.267157 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7c1f-6542-440d-9d48-a623ee0b798a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:58.271006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-44d48ed7-e45c-4a2e-bb01-fa93bf6679ec None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:58.272431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-44d48ed7-e45c-4a2e-bb01-fa93bf6679ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:58.273611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 422/833] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:45:58 2022] GET /identity/v3/role_assignments?user.id=a1dc3ad467d9496cb3e0086a0cd8215e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:58.353805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 412/834] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:45:58.381250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7c1f-6542-440d-9d48-a623ee0b798a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:58.478753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:58.505506 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7c1f-6542-440d-9d48-a623ee0b798a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:58.542587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63a7ac-ee65-4d92-868b-fd271304c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:58.543432 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63a7ac-ee65-4d92-868b-fd271304c188 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:58.543944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63a7ac-ee65-4d92-868b-fd271304c188 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:58.544257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63a7ac-ee65-4d92-868b-fd271304c188 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:58.554898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63a7ac-ee65-4d92-868b-fd271304c188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8d63a7ac-ee65-4d92-868b-fd271304c188 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:45:58.561556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:58.562783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:58.563819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 423/835] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:58.568979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d48ed7-e45c-4a2e-bb01-fa93bf6679ec None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:58.613953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:58.673178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d48ed7-e45c-4a2e-bb01-fa93bf6679ec None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:58.709013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ae59d-20e2-4cfa-91b8-ab4350fa6f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:58.709645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ae59d-20e2-4cfa-91b8-ab4350fa6f7a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:58.709927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ae59d-20e2-4cfa-91b8-ab4350fa6f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:58.710204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ae59d-20e2-4cfa-91b8-ab4350fa6f7a None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:58.710948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ae59d-20e2-4cfa-91b8-ab4350fa6f7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:58.711253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ae59d-20e2-4cfa-91b8-ab4350fa6f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:58.712687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ae59d-20e2-4cfa-91b8-ab4350fa6f7a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:58.721768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 413/836] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:45:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:58.727431 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63a7ac-ee65-4d92-868b-fd271304c188 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:58.834022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63a7ac-ee65-4d92-868b-fd271304c188 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:58.872321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39d485-8b81-4417-9740-c97f11100d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:58.873304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39d485-8b81-4417-9740-c97f11100d1f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:58.873764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39d485-8b81-4417-9740-c97f11100d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:58.874226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39d485-8b81-4417-9740-c97f11100d1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:58.877284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:58.885271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39d485-8b81-4417-9740-c97f11100d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:58.887357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39d485-8b81-4417-9740-c97f11100d1f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1f39d485-8b81-4417-9740-c97f11100d1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:58.903923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:58.904944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:58.905878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:58.905878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:58.905878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:58.905878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:58.905878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 424/837] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:58.912379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ae59d-20e2-4cfa-91b8-ab4350fa6f7a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:59.019849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ae59d-20e2-4cfa-91b8-ab4350fa6f7a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:59.026540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:59.056100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980ab60-c85b-4383-bece-d9cbef0995c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:59.056732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980ab60-c85b-4383-bece-d9cbef0995c1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.057013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980ab60-c85b-4383-bece-d9cbef0995c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.057290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980ab60-c85b-4383-bece-d9cbef0995c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.057981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980ab60-c85b-4383-bece-d9cbef0995c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:59.058282 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980ab60-c85b-4383-bece-d9cbef0995c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:59.060516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980ab60-c85b-4383-bece-d9cbef0995c1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:59.072902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 414/838] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:59.079125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39d485-8b81-4417-9740-c97f11100d1f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:59.187931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39d485-8b81-4417-9740-c97f11100d1f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:59.223891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dee5b-86f4-45da-bab8-12da1d2374de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:59.224542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dee5b-86f4-45da-bab8-12da1d2374de None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.224825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dee5b-86f4-45da-bab8-12da1d2374de None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.225102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dee5b-86f4-45da-bab8-12da1d2374de None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.233288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3dee5b-86f4-45da-bab8-12da1d2374de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:59.234791 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3dee5b-86f4-45da-bab8-12da1d2374de None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ca3dee5b-86f4-45da-bab8-12da1d2374de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:45:59.247025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:59.248027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:59.248984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 425/839] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:45:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:59.254207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980ab60-c85b-4383-bece-d9cbef0995c1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:59.349047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71862-78bf-4546-a81d-151ae424746e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.349375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71862-78bf-4546-a81d-151ae424746e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.349609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71862-78bf-4546-a81d-151ae424746e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.351345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 426/840] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:59.357106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53da90-8f03-489a-a251-148b98953bea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.357391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53da90-8f03-489a-a251-148b98953bea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.357667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53da90-8f03-489a-a251-148b98953bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.368237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980ab60-c85b-4383-bece-d9cbef0995c1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:59.376590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bb53da90-8f03-489a-a251-148b98953bea None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:45:59.377802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb53da90-8f03-489a-a251-148b98953bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:45:59.405740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71385332-6cc6-45f7-ac4e-f2b37ac0e3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:59.406767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71385332-6cc6-45f7-ac4e-f2b37ac0e3ab None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.407328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71385332-6cc6-45f7-ac4e-f2b37ac0e3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.407776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71385332-6cc6-45f7-ac4e-f2b37ac0e3ab None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.408745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71385332-6cc6-45f7-ac4e-f2b37ac0e3ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:59.409225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71385332-6cc6-45f7-ac4e-f2b37ac0e3ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:59.410860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71385332-6cc6-45f7-ac4e-f2b37ac0e3ab None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:59.435920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 415/841] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Jul 30 04:45:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:59.436968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71385332-6cc6-45f7-ac4e-f2b37ac0e3ab None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:59.438868 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:59.462210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 427/842] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:45:59.462998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:59.484797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53da90-8f03-489a-a251-148b98953bea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:59.555474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71385332-6cc6-45f7-ac4e-f2b37ac0e3ab None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:59.598235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cc84f-91ac-472b-9525-5f19029c64f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:59.600463 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cc84f-91ac-472b-9525-5f19029c64f8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.600932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cc84f-91ac-472b-9525-5f19029c64f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.601352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cc84f-91ac-472b-9525-5f19029c64f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.611640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53da90-8f03-489a-a251-148b98953bea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:59.648165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea8a65-de0f-4474-9e61-7bd71b32ee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:59.649085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8a65-de0f-4474-9e61-7bd71b32ee62 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.649507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8a65-de0f-4474-9e61-7bd71b32ee62 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.649921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8a65-de0f-4474-9e61-7bd71b32ee62 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.650731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea8a65-de0f-4474-9e61-7bd71b32ee62 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:59.652652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea8a65-de0f-4474-9e61-7bd71b32ee62 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-54ea8a65-de0f-4474-9e61-7bd71b32ee62 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:59.662140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:59.663323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 30 04:45:59.664152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:45:59.666918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 428/843] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:45:59.673652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea8a65-de0f-4474-9e61-7bd71b32ee62 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:59.688459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cc84f-91ac-472b-9525-5f19029c64f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:59.690419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686cc84f-91ac-472b-9525-5f19029c64f8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:59.717186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:59.719599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:59.786302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea8a65-de0f-4474-9e61-7bd71b32ee62 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:59.806383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 416/844] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:45:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:45:59.807604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cc84f-91ac-472b-9525-5f19029c64f8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:59.824317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c5baa-d916-4759-8cd5-544172fa8c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:59.824870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c5baa-d916-4759-8cd5-544172fa8c58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.825135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c5baa-d916-4759-8cd5-544172fa8c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.825407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c5baa-d916-4759-8cd5-544172fa8c58 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.826082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c5baa-d916-4759-8cd5-544172fa8c58 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:45:59.826385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c5baa-d916-4759-8cd5-544172fa8c58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:59.827893 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c5baa-d916-4759-8cd5-544172fa8c58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:59.832932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 429/845] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:45:59.834369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166aaff-8355-4cf2-8aae-c76a961e4b4c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.834684 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166aaff-8355-4cf2-8aae-c76a961e4b4c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.834955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166aaff-8355-4cf2-8aae-c76a961e4b4c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.836556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 430/846] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:45:59.839517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166aaff-8355-4cf2-8aae-c76a961e4b4c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:45:59.923105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cc84f-91ac-472b-9525-5f19029c64f8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:59.948052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:59.960709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bbaaf-b1fc-46ed-8a49-3e112c8ea0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:45:59.961361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bbaaf-b1fc-46ed-8a49-3e112c8ea0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.961640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bbaaf-b1fc-46ed-8a49-3e112c8ea0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.961911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bbaaf-b1fc-46ed-8a49-3e112c8ea0a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:45:59.963716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166aaff-8355-4cf2-8aae-c76a961e4b4c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:45:59.971925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:45:59.973018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bbaaf-b1fc-46ed-8a49-3e112c8ea0a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:45:59.974593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bbaaf-b1fc-46ed-8a49-3e112c8ea0a3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5f9bbaaf-b1fc-46ed-8a49-3e112c8ea0a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:45:59.990180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:45:59.991145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:45:59.992602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:45:59.992602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:45:59.992602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:45:59.992602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:45:59.992602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 417/847] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:45:59.992889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7ab8c-a3c8-4a1d-8bee-17ea86dbd350 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:45:59.993224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7ab8c-a3c8-4a1d-8bee-17ea86dbd350 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:45:59.993389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7ab8c-a3c8-4a1d-8bee-17ea86dbd350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.008984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b329fb-8bc7-4160-8245-dfd1bfcecbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:00.010251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b329fb-8bc7-4160-8245-dfd1bfcecbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.010251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b329fb-8bc7-4160-8245-dfd1bfcecbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.010251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b329fb-8bc7-4160-8245-dfd1bfcecbe5 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.010856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b329fb-8bc7-4160-8245-dfd1bfcecbe5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:00.011159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b329fb-8bc7-4160-8245-dfd1bfcecbe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:00.013675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b329fb-8bc7-4160-8245-dfd1bfcecbe5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:00.029945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 431/848] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:45:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:00.029945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b329fb-8bc7-4160-8245-dfd1bfcecbe5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:00.033624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9ff7ab8c-a3c8-4a1d-8bee-17ea86dbd350 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:00.035132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9ff7ab8c-a3c8-4a1d-8bee-17ea86dbd350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:00.133776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 418/849] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:45:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:00.158846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7ab8c-a3c8-4a1d-8bee-17ea86dbd350 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:00.165939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b329fb-8bc7-4160-8245-dfd1bfcecbe5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:00.204392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ec968-2e08-4b20-a9b2-662a76843c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:00.205024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ec968-2e08-4b20-a9b2-662a76843c57 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.205303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ec968-2e08-4b20-a9b2-662a76843c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.205582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ec968-2e08-4b20-a9b2-662a76843c57 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.206266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ec968-2e08-4b20-a9b2-662a76843c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:00.206626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ec968-2e08-4b20-a9b2-662a76843c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:00.208084 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ec968-2e08-4b20-a9b2-662a76843c57 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:00.223405 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 432/850] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:00.231335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ec968-2e08-4b20-a9b2-662a76843c57 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:00.277144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:00.289524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7ab8c-a3c8-4a1d-8bee-17ea86dbd350 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:00.292105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:00.330262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254ca4f-6f13-4c03-b005-265fb147b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:00.330866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254ca4f-6f13-4c03-b005-265fb147b5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.331237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254ca4f-6f13-4c03-b005-265fb147b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.331604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254ca4f-6f13-4c03-b005-265fb147b5d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.343900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254ca4f-6f13-4c03-b005-265fb147b5d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5254ca4f-6f13-4c03-b005-265fb147b5d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:00.349369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:00.350478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:46:00.351343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:00.351991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 419/851] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:00.357214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254ca4f-6f13-4c03-b005-265fb147b5d8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:00.362151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ec968-2e08-4b20-a9b2-662a76843c57 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:00.400115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072b569-b7c7-4bb4-9025-cfbe398d01e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:00.401483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072b569-b7c7-4bb4-9025-cfbe398d01e6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.401967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072b569-b7c7-4bb4-9025-cfbe398d01e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.403337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072b569-b7c7-4bb4-9025-cfbe398d01e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.413870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072b569-b7c7-4bb4-9025-cfbe398d01e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:00.416287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072b569-b7c7-4bb4-9025-cfbe398d01e6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a072b569-b7c7-4bb4-9025-cfbe398d01e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:46:00.434329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:00.435542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:00.436454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:46:00.436454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:46:00.436454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:46:00.436454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:46:00.436454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:46:00.436454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:46:00.436454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:00.436454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:00.436454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 433/852] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:00.443069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072b569-b7c7-4bb4-9025-cfbe398d01e6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:00.487494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254ca4f-6f13-4c03-b005-265fb147b5d8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:00.528182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02220102-05df-479f-a6c0-6beefa4db799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:00.529364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02220102-05df-479f-a6c0-6beefa4db799 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.529891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02220102-05df-479f-a6c0-6beefa4db799 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.530396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02220102-05df-479f-a6c0-6beefa4db799 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.531341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02220102-05df-479f-a6c0-6beefa4db799 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:00.531841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02220102-05df-479f-a6c0-6beefa4db799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:00.533425 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02220102-05df-479f-a6c0-6beefa4db799 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:00.544953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 420/853] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:00.546632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e041d8-e5c5-48c9-a24e-31231c6573a8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.547146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e041d8-e5c5-48c9-a24e-31231c6573a8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.547633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e041d8-e5c5-48c9-a24e-31231c6573a8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.549575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 421/854] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:00.550525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e041d8-e5c5-48c9-a24e-31231c6573a8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:00.565312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:00.577570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072b569-b7c7-4bb4-9025-cfbe398d01e6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:00.579434 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:00.615857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb60cf7-efa2-4b8a-afbf-4190c938e940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:00.616481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb60cf7-efa2-4b8a-afbf-4190c938e940 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.616760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb60cf7-efa2-4b8a-afbf-4190c938e940 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.617034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb60cf7-efa2-4b8a-afbf-4190c938e940 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.617776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb60cf7-efa2-4b8a-afbf-4190c938e940 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:00.618081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb60cf7-efa2-4b8a-afbf-4190c938e940 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:00.619546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb60cf7-efa2-4b8a-afbf-4190c938e940 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:00.632384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 434/855] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:00.633606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdd5a0-ca9a-4f8e-a374-c09d11fdd331 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.633894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdd5a0-ca9a-4f8e-a374-c09d11fdd331 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.634169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdd5a0-ca9a-4f8e-a374-c09d11fdd331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.654294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e0cdd5a0-ca9a-4f8e-a374-c09d11fdd331 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:00.655542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e0cdd5a0-ca9a-4f8e-a374-c09d11fdd331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:00.664071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e041d8-e5c5-48c9-a24e-31231c6573a8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:00.700592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086848d-6164-4ed6-aeb1-0f529a41353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:00.701339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086848d-6164-4ed6-aeb1-0f529a41353b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.701621 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086848d-6164-4ed6-aeb1-0f529a41353b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.701887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086848d-6164-4ed6-aeb1-0f529a41353b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.713010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086848d-6164-4ed6-aeb1-0f529a41353b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:00.714470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086848d-6164-4ed6-aeb1-0f529a41353b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:00.726901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 435/856] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:00.727638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdd5a0-ca9a-4f8e-a374-c09d11fdd331 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7086848d-6164-4ed6-aeb1-0f529a41353b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:00.741931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:00.743145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:00.744058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:00.744058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:00.744058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:00.744058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:00.744058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 422/857] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:00.751331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086848d-6164-4ed6-aeb1-0f529a41353b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:00.820913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:00.844203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdd5a0-ca9a-4f8e-a374-c09d11fdd331 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:00.844695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:00.869166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086848d-6164-4ed6-aeb1-0f529a41353b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:00.882284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d0dd3-8bd9-48bf-849c-1daecb259e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:00.882952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d0dd3-8bd9-48bf-849c-1daecb259e69 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.883232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d0dd3-8bd9-48bf-849c-1daecb259e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.883502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d0dd3-8bd9-48bf-849c-1daecb259e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.884279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d0dd3-8bd9-48bf-849c-1daecb259e69 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=54bda2af62ee4ee39a9a350b7ce6430e) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:00.884586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d0dd3-8bd9-48bf-849c-1daecb259e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:00.886016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d0dd3-8bd9-48bf-849c-1daecb259e69 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:00.892584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 436/858] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/role_assignments?user.id=54bda2af62ee4ee39a9a350b7ce6430e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:00.893348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d0dd3-8bd9-48bf-849c-1daecb259e69 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:00.921228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96be854-4488-4b97-81a4-d3ef657ac870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:00.921873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96be854-4488-4b97-81a4-d3ef657ac870 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:00.922148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96be854-4488-4b97-81a4-d3ef657ac870 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:00.922417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96be854-4488-4b97-81a4-d3ef657ac870 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:00.923153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96be854-4488-4b97-81a4-d3ef657ac870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:00.923453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96be854-4488-4b97-81a4-d3ef657ac870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:00.924869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96be854-4488-4b97-81a4-d3ef657ac870 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:00.940283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 423/859] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:00.945677 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96be854-4488-4b97-81a4-d3ef657ac870 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:01.041960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d0dd3-8bd9-48bf-849c-1daecb259e69 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:01.087560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66befb8-b0e0-4e9f-84aa-4a7cccbf506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:01.088242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66befb8-b0e0-4e9f-84aa-4a7cccbf506e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:01.088522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66befb8-b0e0-4e9f-84aa-4a7cccbf506e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:01.088790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66befb8-b0e0-4e9f-84aa-4a7cccbf506e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:01.089415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66befb8-b0e0-4e9f-84aa-4a7cccbf506e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:01.091287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96be854-4488-4b97-81a4-d3ef657ac870 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:01.091408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66befb8-b0e0-4e9f-84aa-4a7cccbf506e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d66befb8-b0e0-4e9f-84aa-4a7cccbf506e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:01.101719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:01.102732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 30 04:46:01.103812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:01.106553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 437/860] 10.222.0.20 () {56 vars in 1132 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:01.109817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66befb8-b0e0-4e9f-84aa-4a7cccbf506e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:01.122420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:01.135743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9168330-5a5a-4bfb-9325-ebbaa892c232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:01.136389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168330-5a5a-4bfb-9325-ebbaa892c232 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:01.137253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168330-5a5a-4bfb-9325-ebbaa892c232 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:01.137530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168330-5a5a-4bfb-9325-ebbaa892c232 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:01.145629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9168330-5a5a-4bfb-9325-ebbaa892c232 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:01.146451 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9168330-5a5a-4bfb-9325-ebbaa892c232 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:01.150642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c9168330-5a5a-4bfb-9325-ebbaa892c232 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:46:01.169970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:01.171198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:46:01.172265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:01.175669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 424/861] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:46:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:01.176921 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9168330-5a5a-4bfb-9325-ebbaa892c232 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:01.250002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66befb8-b0e0-4e9f-84aa-4a7cccbf506e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:01.286964 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9168330-5a5a-4bfb-9325-ebbaa892c232 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:01.294462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b8469-77f7-4b52-bae4-4bfe7468a76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:01.295373 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b8469-77f7-4b52-bae4-4bfe7468a76b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:01.295801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b8469-77f7-4b52-bae4-4bfe7468a76b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:01.296225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b8469-77f7-4b52-bae4-4bfe7468a76b None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:01.297155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b8469-77f7-4b52-bae4-4bfe7468a76b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:01.297607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b8469-77f7-4b52-bae4-4bfe7468a76b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:01.299279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b8469-77f7-4b52-bae4-4bfe7468a76b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:01.304765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 438/862] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:01.311172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b8469-77f7-4b52-bae4-4bfe7468a76b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:01.331180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07cca6-5f1f-4578-a107-0a5d376873b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:01.332394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07cca6-5f1f-4578-a107-0a5d376873b1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:01.333106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07cca6-5f1f-4578-a107-0a5d376873b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:01.333531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07cca6-5f1f-4578-a107-0a5d376873b1 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:01.334933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07cca6-5f1f-4578-a107-0a5d376873b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:01.335912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07cca6-5f1f-4578-a107-0a5d376873b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:01.338848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07cca6-5f1f-4578-a107-0a5d376873b1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:01.365303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 425/863] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Jul 30 04:46:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:01.371315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07cca6-5f1f-4578-a107-0a5d376873b1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:01.410906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:01.434484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b8469-77f7-4b52-bae4-4bfe7468a76b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:01.447467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:01.475804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681137d-0bad-4f5e-952a-a2aded6a76fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:01.476914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681137d-0bad-4f5e-952a-a2aded6a76fe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:01.477293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681137d-0bad-4f5e-952a-a2aded6a76fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:01.477638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681137d-0bad-4f5e-952a-a2aded6a76fe None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:01.478382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681137d-0bad-4f5e-952a-a2aded6a76fe None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:01.478780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681137d-0bad-4f5e-952a-a2aded6a76fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:01.481447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681137d-0bad-4f5e-952a-a2aded6a76fe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:01.490598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 439/864] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:01.520576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07cca6-5f1f-4578-a107-0a5d376873b1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:01.568472 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013af51-b66a-435b-b2a2-ebcef3927c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:01.569433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013af51-b66a-435b-b2a2-ebcef3927c04 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:01.569920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013af51-b66a-435b-b2a2-ebcef3927c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:01.570375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013af51-b66a-435b-b2a2-ebcef3927c04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:01.649422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013af51-b66a-435b-b2a2-ebcef3927c04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:01.651479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013af51-b66a-435b-b2a2-ebcef3927c04 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:01.715651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:01.765880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 426/865] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:01.771360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681137d-0bad-4f5e-952a-a2aded6a76fe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:01.880129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681137d-0bad-4f5e-952a-a2aded6a76fe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:01.921405 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8ce74-4bd1-49f4-805e-ac98192770c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:01.922251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8ce74-4bd1-49f4-805e-ac98192770c3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:01.926550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8ce74-4bd1-49f4-805e-ac98192770c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:01.926550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b8ce74-4bd1-49f4-805e-ac98192770c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:01.937233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8ce74-4bd1-49f4-805e-ac98192770c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:01.938983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b8ce74-4bd1-49f4-805e-ac98192770c3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e6b8ce74-4bd1-49f4-805e-ac98192770c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:01.953948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:01.954902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:01.955820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:01.955820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:01.955820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:01.955820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:01.957958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:01.958391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 440/866] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:01.970460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013af51-b66a-435b-b2a2-ebcef3927c04 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:02.068701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013af51-b66a-435b-b2a2-ebcef3927c04 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:02.104678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93483d-3796-4abd-981f-40ddaa49fa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:02.105356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93483d-3796-4abd-981f-40ddaa49fa54 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:02.105638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93483d-3796-4abd-981f-40ddaa49fa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:02.105907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93483d-3796-4abd-981f-40ddaa49fa54 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:02.106827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93483d-3796-4abd-981f-40ddaa49fa54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:02.107131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93483d-3796-4abd-981f-40ddaa49fa54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:02.108557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93483d-3796-4abd-981f-40ddaa49fa54 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:02.126055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 427/867] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:02.127473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8ce74-4bd1-49f4-805e-ac98192770c3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:02.165484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8281010-5fbb-485f-9195-0367645ffad3 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:02.165767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8281010-5fbb-485f-9195-0367645ffad3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:02.165952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8281010-5fbb-485f-9195-0367645ffad3 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:02.168032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 428/868] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:02.176212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2ff92-48ff-40b1-bbde-596304e3548e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:02.176212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2ff92-48ff-40b1-bbde-596304e3548e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:02.176710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2ff92-48ff-40b1-bbde-596304e3548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:02.196264 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b6c2ff92-48ff-40b1-bbde-596304e3548e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:02.197853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b6c2ff92-48ff-40b1-bbde-596304e3548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:02.215007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:02.242219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b8ce74-4bd1-49f4-805e-ac98192770c3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:02.278547 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 429/869] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:02.282262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03570db5-df92-46aa-beba-ff981ab6b4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:02.283177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03570db5-df92-46aa-beba-ff981ab6b4bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:02.283604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03570db5-df92-46aa-beba-ff981ab6b4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:02.284027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03570db5-df92-46aa-beba-ff981ab6b4bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:02.291352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03570db5-df92-46aa-beba-ff981ab6b4bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:02.293163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03570db5-df92-46aa-beba-ff981ab6b4bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:02.297810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2ff92-48ff-40b1-bbde-596304e3548e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-03570db5-df92-46aa-beba-ff981ab6b4bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:46:02.309786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:02.310976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:02.311935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:46:02.311935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:46:02.311935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:46:02.311935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:46:02.311935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:46:02.311935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:46:02.311935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:02.311935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:02.314516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 441/870] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:02.318791 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03570db5-df92-46aa-beba-ff981ab6b4bc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:02.374832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:02.413827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2ff92-48ff-40b1-bbde-596304e3548e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:02.432532 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03570db5-df92-46aa-beba-ff981ab6b4bc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:02.451967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25900db0-cb92-4445-9d9f-2e50d0449c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:02.452869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25900db0-cb92-4445-9d9f-2e50d0449c9c None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:02.453056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25900db0-cb92-4445-9d9f-2e50d0449c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:02.453438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25900db0-cb92-4445-9d9f-2e50d0449c9c None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:02.454222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25900db0-cb92-4445-9d9f-2e50d0449c9c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:02.455687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25900db0-cb92-4445-9d9f-2e50d0449c9c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:02.472414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b0822-6860-4d1c-b834-5c088cfbd1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:02.473331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b0822-6860-4d1c-b834-5c088cfbd1cf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:02.473796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b0822-6860-4d1c-b834-5c088cfbd1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:02.474276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b0822-6860-4d1c-b834-5c088cfbd1cf None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:02.476282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b0822-6860-4d1c-b834-5c088cfbd1cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:02.476848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b0822-6860-4d1c-b834-5c088cfbd1cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:02.478623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b0822-6860-4d1c-b834-5c088cfbd1cf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:02.479375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 430/871] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:46:02 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:46:02.495017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 442/872] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:46:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:02.503868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25900db0-cb92-4445-9d9f-2e50d0449c9c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:02.514242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:02.627938 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25900db0-cb92-4445-9d9f-2e50d0449c9c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:02.664723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f14985-8534-486e-b000-2a3ca2a1d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:02.665346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f14985-8534-486e-b000-2a3ca2a1d56e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:02.665652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f14985-8534-486e-b000-2a3ca2a1d56e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:02.665889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f14985-8534-486e-b000-2a3ca2a1d56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:02.666711 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f14985-8534-486e-b000-2a3ca2a1d56e None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=e0f2d5364cb84337b77956dc5da78b4d) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:02.667072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f14985-8534-486e-b000-2a3ca2a1d56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:02.669075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f14985-8534-486e-b000-2a3ca2a1d56e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:02.678521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 431/873] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:02 2022] GET /identity/v3/role_assignments?user.id=e0f2d5364cb84337b77956dc5da78b4d&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:03.029143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ed202-f561-47ab-b113-e1bb886895ea None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:03.030051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ed202-f561-47ab-b113-e1bb886895ea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:03.030553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ed202-f561-47ab-b113-e1bb886895ea None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:03.032359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 443/874] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:03.038007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ae144-45f7-48a5-adba-a771faff4d3e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:03.038852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ae144-45f7-48a5-adba-a771faff4d3e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:03.039423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ae144-45f7-48a5-adba-a771faff4d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:03.060498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-652ae144-45f7-48a5-adba-a771faff4d3e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:03.062031 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-652ae144-45f7-48a5-adba-a771faff4d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:03.128501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:03.142741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 432/875] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:03.160433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ed202-f561-47ab-b113-e1bb886895ea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:03.276224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ed202-f561-47ab-b113-e1bb886895ea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:03.289158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:03.318086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12e89f-5783-4250-8cbd-bf95f8531556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:03.319117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12e89f-5783-4250-8cbd-bf95f8531556 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:03.319663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12e89f-5783-4250-8cbd-bf95f8531556 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:03.320149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12e89f-5783-4250-8cbd-bf95f8531556 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:03.320975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12e89f-5783-4250-8cbd-bf95f8531556 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:03.322569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12e89f-5783-4250-8cbd-bf95f8531556 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cb12e89f-5783-4250-8cbd-bf95f8531556 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:03.332823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:03.333822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:03.334707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:03.337469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 444/876] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:03.345373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ae144-45f7-48a5-adba-a771faff4d3e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:03.450471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ae144-45f7-48a5-adba-a771faff4d3e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:03.487118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81156b5-8186-4ed6-b378-ec8d31a7ad53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:03.488089 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81156b5-8186-4ed6-b378-ec8d31a7ad53 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:03.488576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81156b5-8186-4ed6-b378-ec8d31a7ad53 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:03.489049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81156b5-8186-4ed6-b378-ec8d31a7ad53 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:03.489920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81156b5-8186-4ed6-b378-ec8d31a7ad53 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:03.490470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81156b5-8186-4ed6-b378-ec8d31a7ad53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:03.492026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81156b5-8186-4ed6-b378-ec8d31a7ad53 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:03.498270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 433/877] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:03.504883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12e89f-5783-4250-8cbd-bf95f8531556 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:03.626352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12e89f-5783-4250-8cbd-bf95f8531556 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:03.659458 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92d38-6c8d-479b-862f-29e47bd8b20c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:03.659804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92d38-6c8d-479b-862f-29e47bd8b20c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:03.660016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92d38-6c8d-479b-862f-29e47bd8b20c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:03.661724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 434/878] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:03.665264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eeb7b-13d9-4bb6-b95b-c4f07fd0aba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:03.666346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eeb7b-13d9-4bb6-b95b-c4f07fd0aba3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:03.666858 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eeb7b-13d9-4bb6-b95b-c4f07fd0aba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:03.667191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72080152-bc2f-478f-af96-4f020280f71a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:03.667479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72080152-bc2f-478f-af96-4f020280f71a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:03.667833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40eeb7b-13d9-4bb6-b95b-c4f07fd0aba3 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:03.668443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72080152-bc2f-478f-af96-4f020280f71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:03.669765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40eeb7b-13d9-4bb6-b95b-c4f07fd0aba3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:03.670285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40eeb7b-13d9-4bb6-b95b-c4f07fd0aba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:03.671883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40eeb7b-13d9-4bb6-b95b-c4f07fd0aba3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:03.678637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 445/879] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:03.689244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-72080152-bc2f-478f-af96-4f020280f71a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:03.690516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-72080152-bc2f-478f-af96-4f020280f71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:03.755356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:03.768780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 435/880] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:03.792068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eeb7b-13d9-4bb6-b95b-c4f07fd0aba3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:03.831920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:03.912097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40eeb7b-13d9-4bb6-b95b-c4f07fd0aba3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:03.948060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fe81c-fb06-40d5-85d3-f66cdae8c9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:03.948749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fe81c-fb06-40d5-85d3-f66cdae8c9ce None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:03.949101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fe81c-fb06-40d5-85d3-f66cdae8c9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:03.949389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fe81c-fb06-40d5-85d3-f66cdae8c9ce None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:03.950074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fe81c-fb06-40d5-85d3-f66cdae8c9ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:03.951546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fe81c-fb06-40d5-85d3-f66cdae8c9ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:03.969397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 446/881] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:46:03 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:46:04.577801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f864d-e5e7-454e-bca0-86a5c0ab8d04 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:04.578359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f864d-e5e7-454e-bca0-86a5c0ab8d04 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:04.578359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f864d-e5e7-454e-bca0-86a5c0ab8d04 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:04.579969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 436/882] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:04.585017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d53767-0297-4637-ae60-a35e92ce5686 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:04.585403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d53767-0297-4637-ae60-a35e92ce5686 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:04.585567 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d53767-0297-4637-ae60-a35e92ce5686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:04.604041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-86d53767-0297-4637-ae60-a35e92ce5686 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:04.605831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-86d53767-0297-4637-ae60-a35e92ce5686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:04.679310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 447/883] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:04.702916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f864d-e5e7-454e-bca0-86a5c0ab8d04 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:04.809765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f864d-e5e7-454e-bca0-86a5c0ab8d04 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:04.812608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f61d2-8487-4057-94dc-5d5924a4de88 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:04.812906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f61d2-8487-4057-94dc-5d5924a4de88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:04.813174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f61d2-8487-4057-94dc-5d5924a4de88 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:04.815734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 448/884] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:04.819606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a075f5c-8da7-431f-ad9c-720146a99e30 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:04.819886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a075f5c-8da7-431f-ad9c-720146a99e30 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:04.820157 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a075f5c-8da7-431f-ad9c-720146a99e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:04.838601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0a075f5c-8da7-431f-ad9c-720146a99e30 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:04.840330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a075f5c-8da7-431f-ad9c-720146a99e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:04.848776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9bc8f-d8cf-4cbe-94d3-05cc9c842bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:04.849428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9bc8f-d8cf-4cbe-94d3-05cc9c842bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:04.849708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9bc8f-d8cf-4cbe-94d3-05cc9c842bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:04.850046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9bc8f-d8cf-4cbe-94d3-05cc9c842bf5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:04.851084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9bc8f-d8cf-4cbe-94d3-05cc9c842bf5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:04.853123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9bc8f-d8cf-4cbe-94d3-05cc9c842bf5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:04.858787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-04d9bc8f-d8cf-4cbe-94d3-05cc9c842bf5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:04.864734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:04.865759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 30 04:46:04.866646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:04.868368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 437/885] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:46:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:04.873509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9bc8f-d8cf-4cbe-94d3-05cc9c842bf5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:04.937786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 449/886] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:04.951219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:04.960883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a075f5c-8da7-431f-ad9c-720146a99e30 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:05.002619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9bc8f-d8cf-4cbe-94d3-05cc9c842bf5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:05.040573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a553801-a95e-411b-8391-1cb1911ab4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:05.041514 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a553801-a95e-411b-8391-1cb1911ab4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.041992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a553801-a95e-411b-8391-1cb1911ab4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.042491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a553801-a95e-411b-8391-1cb1911ab4e2 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.044035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a553801-a95e-411b-8391-1cb1911ab4e2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:05.044552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a553801-a95e-411b-8391-1cb1911ab4e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:05.046107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a553801-a95e-411b-8391-1cb1911ab4e2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:05.051412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 438/887] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:46:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:05.060620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a553801-a95e-411b-8391-1cb1911ab4e2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:05.090773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a075f5c-8da7-431f-ad9c-720146a99e30 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:05.127418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f61088-1bd4-4d14-b633-ec774167af6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:05.128439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f61088-1bd4-4d14-b633-ec774167af6b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.128932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f61088-1bd4-4d14-b633-ec774167af6b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.129433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f61088-1bd4-4d14-b633-ec774167af6b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.140557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f61088-1bd4-4d14-b633-ec774167af6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-29f61088-1bd4-4d14-b633-ec774167af6b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:05.145948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:05.146965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:05.147890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:05.156677 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 450/888] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Jul 30 04:46:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:05.158522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f61088-1bd4-4d14-b633-ec774167af6b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:05.175534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:05.179934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a553801-a95e-411b-8391-1cb1911ab4e2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:05.202641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:05.221912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931ee73-9226-40af-9df7-f5adf09d6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:05.223038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931ee73-9226-40af-9df7-f5adf09d6de4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.223592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931ee73-9226-40af-9df7-f5adf09d6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.224106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931ee73-9226-40af-9df7-f5adf09d6de4 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.224977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931ee73-9226-40af-9df7-f5adf09d6de4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:05.225472 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931ee73-9226-40af-9df7-f5adf09d6de4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:05.227242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931ee73-9226-40af-9df7-f5adf09d6de4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:05.233601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 439/889] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:46:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:05.239059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931ee73-9226-40af-9df7-f5adf09d6de4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:05.290019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f61088-1bd4-4d14-b633-ec774167af6b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:05.334139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025721e-7843-4fce-be46-00c0903c67c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:05.334804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025721e-7843-4fce-be46-00c0903c67c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.335081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025721e-7843-4fce-be46-00c0903c67c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.335351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025721e-7843-4fce-be46-00c0903c67c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.336067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025721e-7843-4fce-be46-00c0903c67c0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:05.336370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025721e-7843-4fce-be46-00c0903c67c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:05.338151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025721e-7843-4fce-be46-00c0903c67c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:05.349549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 451/890] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Jul 30 04:46:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:05.356052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025721e-7843-4fce-be46-00c0903c67c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:05.358977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931ee73-9226-40af-9df7-f5adf09d6de4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:05.401510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4adb9-2d6d-4f32-9698-d63ea1da12f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:05.402239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4adb9-2d6d-4f32-9698-d63ea1da12f6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.402532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4adb9-2d6d-4f32-9698-d63ea1da12f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.402833 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4adb9-2d6d-4f32-9698-d63ea1da12f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.403709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4adb9-2d6d-4f32-9698-d63ea1da12f6 None admin] RBAC: Adding query filter params (interface=public, service_id=21b1a38253ed4813b3a14a5fee0aea60, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:05.404016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4adb9-2d6d-4f32-9698-d63ea1da12f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:05.405755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4adb9-2d6d-4f32-9698-d63ea1da12f6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:05.414584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 440/891] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Jul 30 04:46:05 2022] GET /identity/v3/endpoints?service_id=21b1a38253ed4813b3a14a5fee0aea60&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:05.419419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4adb9-2d6d-4f32-9698-d63ea1da12f6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:05.445980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:05.475360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025721e-7843-4fce-be46-00c0903c67c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:05.476845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:05.519334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f943669-fbb0-4ebf-acda-bb25ebc1150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:05.520433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f943669-fbb0-4ebf-acda-bb25ebc1150f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.520948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f943669-fbb0-4ebf-acda-bb25ebc1150f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.521427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f943669-fbb0-4ebf-acda-bb25ebc1150f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.534834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f943669-fbb0-4ebf-acda-bb25ebc1150f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:05.536616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f943669-fbb0-4ebf-acda-bb25ebc1150f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:05.545300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4adb9-2d6d-4f32-9698-d63ea1da12f6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3f943669-fbb0-4ebf-acda-bb25ebc1150f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:05.555969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:05.556891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:05.557805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:05.557805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:05.557805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:05.557805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:05.561193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 452/892] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:05.567393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f943669-fbb0-4ebf-acda-bb25ebc1150f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:05.585120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95faa141-a571-4941-8de3-f7fa0b4d227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:05.586512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faa141-a571-4941-8de3-f7fa0b4d227c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.587000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faa141-a571-4941-8de3-f7fa0b4d227c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.587472 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95faa141-a571-4941-8de3-f7fa0b4d227c None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.588447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95faa141-a571-4941-8de3-f7fa0b4d227c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:05.590728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95faa141-a571-4941-8de3-f7fa0b4d227c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:05.598255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 441/893] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Jul 30 04:46:05 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:46:05.697468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f943669-fbb0-4ebf-acda-bb25ebc1150f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:05.725051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:05.737255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31731d3c-fc08-4d1d-8b37-6e5e6e89a6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:05.738287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31731d3c-fc08-4d1d-8b37-6e5e6e89a6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.738852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31731d3c-fc08-4d1d-8b37-6e5e6e89a6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.739338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31731d3c-fc08-4d1d-8b37-6e5e6e89a6c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.740280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31731d3c-fc08-4d1d-8b37-6e5e6e89a6c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:05.740801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31731d3c-fc08-4d1d-8b37-6e5e6e89a6c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:05.742674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31731d3c-fc08-4d1d-8b37-6e5e6e89a6c1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:05.761846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 453/894] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:05.767676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95faa141-a571-4941-8de3-f7fa0b4d227c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:05.851195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218222e4-00f8-41ea-be7e-768637f16161 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.851522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218222e4-00f8-41ea-be7e-768637f16161 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.851766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218222e4-00f8-41ea-be7e-768637f16161 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.853375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 454/895] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:05.859225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da9f6b-2626-4407-b7e7-1ccecbac8a22 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.859511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da9f6b-2626-4407-b7e7-1ccecbac8a22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.859783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da9f6b-2626-4407-b7e7-1ccecbac8a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.878844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-80da9f6b-2626-4407-b7e7-1ccecbac8a22 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:05.880055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-80da9f6b-2626-4407-b7e7-1ccecbac8a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:05.881340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95faa141-a571-4941-8de3-f7fa0b4d227c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:05.896800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:05.929337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439684fc-aad7-4137-993b-b8c2ca91a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:05.929980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439684fc-aad7-4137-993b-b8c2ca91a0ce None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:05.930258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439684fc-aad7-4137-993b-b8c2ca91a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:05.930704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439684fc-aad7-4137-993b-b8c2ca91a0ce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:05.941406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439684fc-aad7-4137-993b-b8c2ca91a0ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:05.943507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439684fc-aad7-4137-993b-b8c2ca91a0ce None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:05.956329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 455/896] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-439684fc-aad7-4137-993b-b8c2ca91a0ce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:46:05.958053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:05.959208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:46:05.960439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:05.966906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 442/897] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:46:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:05.968675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da9f6b-2626-4407-b7e7-1ccecbac8a22 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:05.979257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439684fc-aad7-4137-993b-b8c2ca91a0ce None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:06.079743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da9f6b-2626-4407-b7e7-1ccecbac8a22 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:06.098180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:06.104195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439684fc-aad7-4137-993b-b8c2ca91a0ce None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:06.118743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24d62c-c9f5-4016-ad6e-7ceb32980b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:06.119364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d62c-c9f5-4016-ad6e-7ceb32980b90 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.119642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d62c-c9f5-4016-ad6e-7ceb32980b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.119912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d62c-c9f5-4016-ad6e-7ceb32980b90 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.121534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24d62c-c9f5-4016-ad6e-7ceb32980b90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:06.121860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24d62c-c9f5-4016-ad6e-7ceb32980b90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:06.123416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24d62c-c9f5-4016-ad6e-7ceb32980b90 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:06.139330 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a1641-f93f-4a2c-8f6a-d5779a782606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:06.139952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a1641-f93f-4a2c-8f6a-d5779a782606 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.140234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a1641-f93f-4a2c-8f6a-d5779a782606 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.140518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a1641-f93f-4a2c-8f6a-d5779a782606 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.141207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a1641-f93f-4a2c-8f6a-d5779a782606 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:06.142603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a1641-f93f-4a2c-8f6a-d5779a782606 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:06.147266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 456/898] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Jul 30 04:46:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:06.147613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c89d27-1877-4d41-9b43-4d4163c8ad47 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.147799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c89d27-1877-4d41-9b43-4d4163c8ad47 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.148076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c89d27-1877-4d41-9b43-4d4163c8ad47 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.149379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 457/899] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:06.150077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c89d27-1877-4d41-9b43-4d4163c8ad47 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:06.165270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 443/900] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:46:05 2022] POST /identity/v3/services => generated 241 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:06.166392 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ff8e5-1125-49a1-9203-d885de326b71 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.170801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ff8e5-1125-49a1-9203-d885de326b71 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.171820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ff8e5-1125-49a1-9203-d885de326b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.200150 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d89ff8e5-1125-49a1-9203-d885de326b71 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:06.202542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d89ff8e5-1125-49a1-9203-d885de326b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:06.242341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:06.280644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c89d27-1877-4d41-9b43-4d4163c8ad47 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:06.313113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 444/901] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:06.324783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289426f1-f535-42d2-8712-128c8696fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:06.325416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289426f1-f535-42d2-8712-128c8696fb04 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.325697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289426f1-f535-42d2-8712-128c8696fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.325992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289426f1-f535-42d2-8712-128c8696fb04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.336131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ff8e5-1125-49a1-9203-d885de326b71 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:06.370328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:06.409830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289426f1-f535-42d2-8712-128c8696fb04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:06.411363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289426f1-f535-42d2-8712-128c8696fb04 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:06.474930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ff8e5-1125-49a1-9203-d885de326b71 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:06.526458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 458/902] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:06 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:06.528263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662d7a3-c204-4a48-8d35-09f00126169c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:06.528849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289426f1-f535-42d2-8712-128c8696fb04 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:06.528958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662d7a3-c204-4a48-8d35-09f00126169c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.529192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662d7a3-c204-4a48-8d35-09f00126169c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.529465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662d7a3-c204-4a48-8d35-09f00126169c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.530099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662d7a3-c204-4a48-8d35-09f00126169c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:06.531515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662d7a3-c204-4a48-8d35-09f00126169c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6662d7a3-c204-4a48-8d35-09f00126169c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:06.541127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:06.542112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:06.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 445/903] 10.222.0.20 () {56 vars in 1132 bytes} [Sat Jul 30 04:46:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:06.548596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662d7a3-c204-4a48-8d35-09f00126169c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:06.576272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:06.625332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:06.660325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662d7a3-c204-4a48-8d35-09f00126169c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:06.667567 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289426f1-f535-42d2-8712-128c8696fb04 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:06.699136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3ffc5-f0ee-4ce9-b916-6dd55110e3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:06.699936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3ffc5-f0ee-4ce9-b916-6dd55110e3cf None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.700227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3ffc5-f0ee-4ce9-b916-6dd55110e3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.700504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3ffc5-f0ee-4ce9-b916-6dd55110e3cf None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.701294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3ffc5-f0ee-4ce9-b916-6dd55110e3cf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:06.701653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3ffc5-f0ee-4ce9-b916-6dd55110e3cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:06.703066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3ffc5-f0ee-4ce9-b916-6dd55110e3cf None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:06.713090 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c03e6a-2253-48a2-bc48-5ce1364e0d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:06.713568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 446/904] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:06.714158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3ffc5-f0ee-4ce9-b916-6dd55110e3cf None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:06.715535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c03e6a-2253-48a2-bc48-5ce1364e0d57 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.715993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c03e6a-2253-48a2-bc48-5ce1364e0d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.716436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c03e6a-2253-48a2-bc48-5ce1364e0d57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.729501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c03e6a-2253-48a2-bc48-5ce1364e0d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:06.731573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c03e6a-2253-48a2-bc48-5ce1364e0d57 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-29c03e6a-2253-48a2-bc48-5ce1364e0d57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:06.751579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:06.752494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:06.753383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:06.753383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:06.753383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:06.753383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:06.760846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 459/905] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:06.762062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c03e6a-2253-48a2-bc48-5ce1364e0d57 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:06.815193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:06.831697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3ffc5-f0ee-4ce9-b916-6dd55110e3cf None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:06.875892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35b794-0fcd-4e0a-9ed6-f352ba3492f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:06.877171 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35b794-0fcd-4e0a-9ed6-f352ba3492f8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.877596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35b794-0fcd-4e0a-9ed6-f352ba3492f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.878028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35b794-0fcd-4e0a-9ed6-f352ba3492f8 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.878888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35b794-0fcd-4e0a-9ed6-f352ba3492f8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:06.879332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35b794-0fcd-4e0a-9ed6-f352ba3492f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:06.881560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35b794-0fcd-4e0a-9ed6-f352ba3492f8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:06.890078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 447/906] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:06.894065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c03e6a-2253-48a2-bc48-5ce1364e0d57 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:06.895694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35b794-0fcd-4e0a-9ed6-f352ba3492f8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:06.925431 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:06.941570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f4938-29c2-4512-95b2-66335d326536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:06.942195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f4938-29c2-4512-95b2-66335d326536 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:06.942506 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f4938-29c2-4512-95b2-66335d326536 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:06.942786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f4938-29c2-4512-95b2-66335d326536 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:06.943476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f4938-29c2-4512-95b2-66335d326536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:06.943779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f4938-29c2-4512-95b2-66335d326536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:06.945148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f4938-29c2-4512-95b2-66335d326536 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:06.965768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 460/907] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:06.967070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f4938-29c2-4512-95b2-66335d326536 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:07.035704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35b794-0fcd-4e0a-9ed6-f352ba3492f8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:07.082954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75435db-ec28-475a-8997-914bd15fd598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:07.083906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75435db-ec28-475a-8997-914bd15fd598 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:07.084394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75435db-ec28-475a-8997-914bd15fd598 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:07.084859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75435db-ec28-475a-8997-914bd15fd598 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:07.086311 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75435db-ec28-475a-8997-914bd15fd598 None admin] RBAC: Adding query filter params (interface=public, service_id=ffe3336716874c848957832bc4d90aa2, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:07.087084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75435db-ec28-475a-8997-914bd15fd598 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:07.089450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75435db-ec28-475a-8997-914bd15fd598 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:07.097156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 448/908] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Jul 30 04:46:06 2022] GET /identity/v3/endpoints?service_id=ffe3336716874c848957832bc4d90aa2&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:07.102034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f4938-29c2-4512-95b2-66335d326536 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:07.102825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75435db-ec28-475a-8997-914bd15fd598 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:07.127425 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:07.148483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df2da-f750-4def-8ae4-4c9d5363dd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:07.149131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df2da-f750-4def-8ae4-4c9d5363dd55 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:07.149410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df2da-f750-4def-8ae4-4c9d5363dd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:07.149683 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df2da-f750-4def-8ae4-4c9d5363dd55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:07.159634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df2da-f750-4def-8ae4-4c9d5363dd55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:07.161814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df2da-f750-4def-8ae4-4c9d5363dd55 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b45df2da-f750-4def-8ae4-4c9d5363dd55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:46:07.179043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:07.179951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:07.180844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:46:07.180844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:46:07.180844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:46:07.180844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:46:07.180844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:46:07.180844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:46:07.180844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:07.180844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:07.183312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 461/909] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:07.189264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df2da-f750-4def-8ae4-4c9d5363dd55 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:07.231853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75435db-ec28-475a-8997-914bd15fd598 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:07.238814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:07.267239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d524ba2-d734-4a2e-a41c-7e4ca09c50e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:07.267957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d524ba2-d734-4a2e-a41c-7e4ca09c50e8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:07.268337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d524ba2-d734-4a2e-a41c-7e4ca09c50e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:07.268695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d524ba2-d734-4a2e-a41c-7e4ca09c50e8 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:07.269424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d524ba2-d734-4a2e-a41c-7e4ca09c50e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:07.270945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d524ba2-d734-4a2e-a41c-7e4ca09c50e8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:07.285531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 449/910] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Jul 30 04:46:07 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:46:07.313159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df2da-f750-4def-8ae4-4c9d5363dd55 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:07.358618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65932e2c-8ebd-4494-a390-b507870aa0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:07.359187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65932e2c-8ebd-4494-a390-b507870aa0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:07.359470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65932e2c-8ebd-4494-a390-b507870aa0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:07.359746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65932e2c-8ebd-4494-a390-b507870aa0d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:07.360525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65932e2c-8ebd-4494-a390-b507870aa0d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:07.360908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65932e2c-8ebd-4494-a390-b507870aa0d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:07.362468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65932e2c-8ebd-4494-a390-b507870aa0d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:07.377585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 462/911] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:46:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:07.383291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d524ba2-d734-4a2e-a41c-7e4ca09c50e8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:07.472187 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:07.492975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d524ba2-d734-4a2e-a41c-7e4ca09c50e8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:07.530364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b363bf6-3d36-4b27-9d60-e5a275d3a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:07.531022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b363bf6-3d36-4b27-9d60-e5a275d3a998 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:07.531299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b363bf6-3d36-4b27-9d60-e5a275d3a998 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:07.531573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b363bf6-3d36-4b27-9d60-e5a275d3a998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:07.532355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b363bf6-3d36-4b27-9d60-e5a275d3a998 None admin] RBAC: Adding query filter params (role.id=be2caee164e146c9ab3f7cd86636da45, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=e0f2d5364cb84337b77956dc5da78b4d) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:07.532659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b363bf6-3d36-4b27-9d60-e5a275d3a998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:07.534080 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b363bf6-3d36-4b27-9d60-e5a275d3a998 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:07.540025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 450/912] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:07 2022] GET /identity/v3/role_assignments?user.id=e0f2d5364cb84337b77956dc5da78b4d&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=be2caee164e146c9ab3f7cd86636da45 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:07.746703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af92f-575a-4d2d-b04d-325ab6a94417 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:07.747255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af92f-575a-4d2d-b04d-325ab6a94417 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:07.747255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af92f-575a-4d2d-b04d-325ab6a94417 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:07.748856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 463/913] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:07.754288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff0988-a870-4053-8460-06e22ba1b316 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:07.754728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff0988-a870-4053-8460-06e22ba1b316 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:07.754887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff0988-a870-4053-8460-06e22ba1b316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:07.775225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-58ff0988-a870-4053-8460-06e22ba1b316 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:07.776699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-58ff0988-a870-4053-8460-06e22ba1b316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:07.854569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 451/914] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:07.872096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637af92f-575a-4d2d-b04d-325ab6a94417 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:07.981047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637af92f-575a-4d2d-b04d-325ab6a94417 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:07.982685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:08.027739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d38d38-3d5b-498a-a222-2df169d356f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:08.028406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d38d38-3d5b-498a-a222-2df169d356f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:08.028685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d38d38-3d5b-498a-a222-2df169d356f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:08.028959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d38d38-3d5b-498a-a222-2df169d356f1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:08.029588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d38d38-3d5b-498a-a222-2df169d356f1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:08.031042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d38d38-3d5b-498a-a222-2df169d356f1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b7d38d38-3d5b-498a-a222-2df169d356f1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:08.040609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:08.041606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:08.042731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 464/915] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:46:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:08.048819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff0988-a870-4053-8460-06e22ba1b316 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:08.129691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:08.156728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff0988-a870-4053-8460-06e22ba1b316 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:08.192923 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a5d00-bc2d-405b-a04f-a9b8e41eb980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:08.193552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a5d00-bc2d-405b-a04f-a9b8e41eb980 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:08.193830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a5d00-bc2d-405b-a04f-a9b8e41eb980 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:08.194104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a5d00-bc2d-405b-a04f-a9b8e41eb980 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:08.194813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a5d00-bc2d-405b-a04f-a9b8e41eb980 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:08.195116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a5d00-bc2d-405b-a04f-a9b8e41eb980 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:08.196482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a5d00-bc2d-405b-a04f-a9b8e41eb980 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:08.202140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 452/916] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:46:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:08.207775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d38d38-3d5b-498a-a222-2df169d356f1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:08.311053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6162f6c-5514-467b-89f3-c0700d3e8c0c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:08.311546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6162f6c-5514-467b-89f3-c0700d3e8c0c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:08.311975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6162f6c-5514-467b-89f3-c0700d3e8c0c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:08.313691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 453/917] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:08.318575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87287b6e-c00a-4aa7-b69d-a724cec8faa9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:08.319004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87287b6e-c00a-4aa7-b69d-a724cec8faa9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:08.319415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87287b6e-c00a-4aa7-b69d-a724cec8faa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:08.331804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d38d38-3d5b-498a-a222-2df169d356f1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:08.343090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-87287b6e-c00a-4aa7-b69d-a724cec8faa9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:08.344357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-87287b6e-c00a-4aa7-b69d-a724cec8faa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:08.369481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22039b-a8ee-49cd-93f5-8c94c711cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:08.370227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22039b-a8ee-49cd-93f5-8c94c711cf80 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:08.370689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22039b-a8ee-49cd-93f5-8c94c711cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:08.371031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22039b-a8ee-49cd-93f5-8c94c711cf80 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:08.371775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22039b-a8ee-49cd-93f5-8c94c711cf80 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:08.372144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22039b-a8ee-49cd-93f5-8c94c711cf80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:08.373751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22039b-a8ee-49cd-93f5-8c94c711cf80 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:08.382534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 465/918] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Jul 30 04:46:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:08.384816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22039b-a8ee-49cd-93f5-8c94c711cf80 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:08.418539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:08.438509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 454/919] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:08.462620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87287b6e-c00a-4aa7-b69d-a724cec8faa9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:08.503416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22039b-a8ee-49cd-93f5-8c94c711cf80 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:08.542409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:08.542631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017880f-b657-4e3c-8f50-225cc2024379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:08.543538 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017880f-b657-4e3c-8f50-225cc2024379 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:08.544195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017880f-b657-4e3c-8f50-225cc2024379 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:08.544195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017880f-b657-4e3c-8f50-225cc2024379 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:08.545259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017880f-b657-4e3c-8f50-225cc2024379 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:08.546900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017880f-b657-4e3c-8f50-225cc2024379 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:08.578618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 466/920] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:46:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:08.590818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87287b6e-c00a-4aa7-b69d-a724cec8faa9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:08.638051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d1fc8-1b15-4b44-a83b-23f77556a069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:08.638749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d1fc8-1b15-4b44-a83b-23f77556a069 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:08.639028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d1fc8-1b15-4b44-a83b-23f77556a069 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:08.639299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d1fc8-1b15-4b44-a83b-23f77556a069 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:08.639942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d1fc8-1b15-4b44-a83b-23f77556a069 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:08.641358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d1fc8-1b15-4b44-a83b-23f77556a069 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-bc5d1fc8-1b15-4b44-a83b-23f77556a069 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:08.651802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:08.653564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:08.654639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 455/921] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:08.661708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017880f-b657-4e3c-8f50-225cc2024379 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:08.764816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:08.786679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017880f-b657-4e3c-8f50-225cc2024379 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:08.821765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe47a3-06a9-4710-ae06-11aaa64d7dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:08.822384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe47a3-06a9-4710-ae06-11aaa64d7dbf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:08.822690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe47a3-06a9-4710-ae06-11aaa64d7dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:08.822958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe47a3-06a9-4710-ae06-11aaa64d7dbf None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:08.824227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe47a3-06a9-4710-ae06-11aaa64d7dbf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:08.824528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe47a3-06a9-4710-ae06-11aaa64d7dbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:08.825898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe47a3-06a9-4710-ae06-11aaa64d7dbf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:08.831371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 467/922] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:08.836870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d1fc8-1b15-4b44-a83b-23f77556a069 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:08.950944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d1fc8-1b15-4b44-a83b-23f77556a069 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:08.978525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:08.987755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2392e7f-554f-4dd6-9f95-695ce7925bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:08.988477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392e7f-554f-4dd6-9f95-695ce7925bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:08.988753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392e7f-554f-4dd6-9f95-695ce7925bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:08.989018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2392e7f-554f-4dd6-9f95-695ce7925bf6 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:08.989690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2392e7f-554f-4dd6-9f95-695ce7925bf6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:08.989986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2392e7f-554f-4dd6-9f95-695ce7925bf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:08.991385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2392e7f-554f-4dd6-9f95-695ce7925bf6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:08.996764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 456/923] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:09.002716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe47a3-06a9-4710-ae06-11aaa64d7dbf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:09.112410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe47a3-06a9-4710-ae06-11aaa64d7dbf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:09.148315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45615e8-81f8-48bf-93ee-335a0a41d9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:09.149308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45615e8-81f8-48bf-93ee-335a0a41d9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:09.149788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45615e8-81f8-48bf-93ee-335a0a41d9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:09.150357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45615e8-81f8-48bf-93ee-335a0a41d9f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:09.152755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45615e8-81f8-48bf-93ee-335a0a41d9f1 None admin] RBAC: Adding query filter params (interface=public, service_id=82ebd465bd2f42469f086d3dc44c0dd1, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:09.153281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45615e8-81f8-48bf-93ee-335a0a41d9f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:09.154872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45615e8-81f8-48bf-93ee-335a0a41d9f1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:09.160852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 468/924] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Jul 30 04:46:09 2022] GET /identity/v3/endpoints?service_id=82ebd465bd2f42469f086d3dc44c0dd1&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:09.170491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2392e7f-554f-4dd6-9f95-695ce7925bf6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:09.279861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2392e7f-554f-4dd6-9f95-695ce7925bf6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:09.315949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aead83-ff5a-4a41-89a7-eadd9ce37f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:09.316928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aead83-ff5a-4a41-89a7-eadd9ce37f3b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:09.317390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aead83-ff5a-4a41-89a7-eadd9ce37f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:09.317869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aead83-ff5a-4a41-89a7-eadd9ce37f3b None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:09.318722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aead83-ff5a-4a41-89a7-eadd9ce37f3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:09.320269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aead83-ff5a-4a41-89a7-eadd9ce37f3b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:09.327366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 457/925] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Jul 30 04:46:09 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:46:09.435503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99299adb-fdc3-42bb-abd2-3a53d0da9634 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:09.435704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99299adb-fdc3-42bb-abd2-3a53d0da9634 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:09.435984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99299adb-fdc3-42bb-abd2-3a53d0da9634 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:09.437672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 469/926] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:09.443312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f7da5-29f1-48ac-80b4-88a673b0dd88 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:09.443599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f7da5-29f1-48ac-80b4-88a673b0dd88 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:09.443880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f7da5-29f1-48ac-80b4-88a673b0dd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:09.463713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a96f7da5-29f1-48ac-80b4-88a673b0dd88 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:09.465562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a96f7da5-29f1-48ac-80b4-88a673b0dd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:09.523024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:09.543978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 458/927] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:09.565880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99299adb-fdc3-42bb-abd2-3a53d0da9634 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:09.583970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:09.667991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352ca0c-8555-4bf4-8e68-81f9922d40db None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:09.668267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352ca0c-8555-4bf4-8e68-81f9922d40db None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:09.668542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352ca0c-8555-4bf4-8e68-81f9922d40db None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:09.670083 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 459/928] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:09.676190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47a1f9-9d94-4c1b-bd3e-eb422146b401 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:09.676467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47a1f9-9d94-4c1b-bd3e-eb422146b401 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:09.676737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47a1f9-9d94-4c1b-bd3e-eb422146b401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:09.690579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99299adb-fdc3-42bb-abd2-3a53d0da9634 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:09.698061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4c47a1f9-9d94-4c1b-bd3e-eb422146b401 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:09.700376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4c47a1f9-9d94-4c1b-bd3e-eb422146b401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:09.737618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cea311-e6e9-4176-99ae-025d0d63df3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:09.739268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cea311-e6e9-4176-99ae-025d0d63df3f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:09.739827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cea311-e6e9-4176-99ae-025d0d63df3f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:09.740353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cea311-e6e9-4176-99ae-025d0d63df3f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:09.741390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cea311-e6e9-4176-99ae-025d0d63df3f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:09.743047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cea311-e6e9-4176-99ae-025d0d63df3f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-99cea311-e6e9-4176-99ae-025d0d63df3f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:09.756466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:09.757665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 30 04:46:09.758768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:09.766620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 470/929] 10.222.0.20 () {56 vars in 1132 bytes} [Sat Jul 30 04:46:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:09.768749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cea311-e6e9-4176-99ae-025d0d63df3f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:09.794500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 460/930] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:09.819734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47a1f9-9d94-4c1b-bd3e-eb422146b401 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:09.874543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:09.907452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cea311-e6e9-4176-99ae-025d0d63df3f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:09.945343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538e312-76a8-4307-9b2e-9efecba2d3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:09.945892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538e312-76a8-4307-9b2e-9efecba2d3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:09.946174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538e312-76a8-4307-9b2e-9efecba2d3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:09.946465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538e312-76a8-4307-9b2e-9efecba2d3a7 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:09.947253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538e312-76a8-4307-9b2e-9efecba2d3a7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:09.947562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538e312-76a8-4307-9b2e-9efecba2d3a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:09.948983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538e312-76a8-4307-9b2e-9efecba2d3a7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:09.954735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 471/931] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:09.960092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538e312-76a8-4307-9b2e-9efecba2d3a7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:09.963770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47a1f9-9d94-4c1b-bd3e-eb422146b401 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:09.982389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:10.004372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02017b16-fe0e-4405-994f-35d3344b0609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:10.005049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02017b16-fe0e-4405-994f-35d3344b0609 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:10.005331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02017b16-fe0e-4405-994f-35d3344b0609 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:10.005603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02017b16-fe0e-4405-994f-35d3344b0609 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:10.017581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02017b16-fe0e-4405-994f-35d3344b0609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-02017b16-fe0e-4405-994f-35d3344b0609 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:10.022883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:10.023861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:10.024735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:10.029804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 461/932] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Jul 30 04:46:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:10.030686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02017b16-fe0e-4405-994f-35d3344b0609 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:10.091464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538e312-76a8-4307-9b2e-9efecba2d3a7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:10.129117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9f1f3-fafb-4275-be8f-2cfd7bf630bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:10.129743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9f1f3-fafb-4275-be8f-2cfd7bf630bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:10.130127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9f1f3-fafb-4275-be8f-2cfd7bf630bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:10.130410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9f1f3-fafb-4275-be8f-2cfd7bf630bb None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:10.131117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9f1f3-fafb-4275-be8f-2cfd7bf630bb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:10.131422 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9f1f3-fafb-4275-be8f-2cfd7bf630bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:10.132884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9f1f3-fafb-4275-be8f-2cfd7bf630bb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:10.138553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 472/933] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:10.144458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9f1f3-fafb-4275-be8f-2cfd7bf630bb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:10.149605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02017b16-fe0e-4405-994f-35d3344b0609 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:10.153970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:10.199529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869754a8-8790-4625-8983-5b363a4ff11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:10.200655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869754a8-8790-4625-8983-5b363a4ff11c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:10.201572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869754a8-8790-4625-8983-5b363a4ff11c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:10.202359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869754a8-8790-4625-8983-5b363a4ff11c None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:10.203267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869754a8-8790-4625-8983-5b363a4ff11c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:10.203995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869754a8-8790-4625-8983-5b363a4ff11c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:10.205965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869754a8-8790-4625-8983-5b363a4ff11c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:10.223256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 462/934] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Jul 30 04:46:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:10.224613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869754a8-8790-4625-8983-5b363a4ff11c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:10.272148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9f1f3-fafb-4275-be8f-2cfd7bf630bb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:10.278152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:10.324720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1357b77-38e2-4113-9151-dffc8ba3eeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:10.325750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1357b77-38e2-4113-9151-dffc8ba3eeac None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:10.326498 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1357b77-38e2-4113-9151-dffc8ba3eeac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:10.326961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1357b77-38e2-4113-9151-dffc8ba3eeac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:10.328264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1357b77-38e2-4113-9151-dffc8ba3eeac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:10.329825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1357b77-38e2-4113-9151-dffc8ba3eeac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:10.352418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869754a8-8790-4625-8983-5b363a4ff11c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:10.374791 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 473/935] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:46:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:10.406225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed836ad-32a9-47e7-a537-0cf68802e045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:10.406925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed836ad-32a9-47e7-a537-0cf68802e045 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:10.407202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed836ad-32a9-47e7-a537-0cf68802e045 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:10.407471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed836ad-32a9-47e7-a537-0cf68802e045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:10.420473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed836ad-32a9-47e7-a537-0cf68802e045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:10.421954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed836ad-32a9-47e7-a537-0cf68802e045 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3ed836ad-32a9-47e7-a537-0cf68802e045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:10.437679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:10.438708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:10.439731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:10.439731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:10.439731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:10.439731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:10.439731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 463/936] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:10.447660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1357b77-38e2-4113-9151-dffc8ba3eeac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:10.528871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:10.575190 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1357b77-38e2-4113-9151-dffc8ba3eeac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:10.610384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47cf79-2c6b-4690-bc38-df13e0d3fd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:10.611463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47cf79-2c6b-4690-bc38-df13e0d3fd69 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:10.611952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47cf79-2c6b-4690-bc38-df13e0d3fd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:10.612457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47cf79-2c6b-4690-bc38-df13e0d3fd69 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:10.613353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47cf79-2c6b-4690-bc38-df13e0d3fd69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:10.613862 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47cf79-2c6b-4690-bc38-df13e0d3fd69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:10.615467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47cf79-2c6b-4690-bc38-df13e0d3fd69 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:10.629358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 474/937] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:10.635158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed836ad-32a9-47e7-a537-0cf68802e045 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:10.729469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:10.746909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed836ad-32a9-47e7-a537-0cf68802e045 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:10.784366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc640e-46c8-404b-af91-5646658bf713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:10.785345 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc640e-46c8-404b-af91-5646658bf713 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:10.785845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc640e-46c8-404b-af91-5646658bf713 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:10.786331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc640e-46c8-404b-af91-5646658bf713 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:10.795384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc640e-46c8-404b-af91-5646658bf713 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:10.797071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc640e-46c8-404b-af91-5646658bf713 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4adc640e-46c8-404b-af91-5646658bf713 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:46:10.809663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:10.810690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:46:10.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:10.814273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 464/938] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:46:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:10.820283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47cf79-2c6b-4690-bc38-df13e0d3fd69 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:10.858343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8420f87-3790-4d55-95fb-90f4e2f7db22 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:10.858683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8420f87-3790-4d55-95fb-90f4e2f7db22 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:10.858946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8420f87-3790-4d55-95fb-90f4e2f7db22 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:10.860432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 465/939] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:10.865401 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da4022-7185-4f1a-a2d5-b7192bc59d7b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:10.865683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da4022-7185-4f1a-a2d5-b7192bc59d7b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:10.865959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da4022-7185-4f1a-a2d5-b7192bc59d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:10.884420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-52da4022-7185-4f1a-a2d5-b7192bc59d7b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:10.885625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-52da4022-7185-4f1a-a2d5-b7192bc59d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:10.954788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47cf79-2c6b-4690-bc38-df13e0d3fd69 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:10.970559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 466/940] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:10.982945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:10.987803 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da4022-7185-4f1a-a2d5-b7192bc59d7b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:10.998714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37608341-6a23-45b7-9df2-2942538a6280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:10.999606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37608341-6a23-45b7-9df2-2942538a6280 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.000054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37608341-6a23-45b7-9df2-2942538a6280 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.000919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37608341-6a23-45b7-9df2-2942538a6280 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.002290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37608341-6a23-45b7-9df2-2942538a6280 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:11.002780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37608341-6a23-45b7-9df2-2942538a6280 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.004711 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37608341-6a23-45b7-9df2-2942538a6280 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.029674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 475/941] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Jul 30 04:46:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:11.035919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37608341-6a23-45b7-9df2-2942538a6280 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:11.081777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:11.109770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da4022-7185-4f1a-a2d5-b7192bc59d7b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:11.144457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37608341-6a23-45b7-9df2-2942538a6280 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:11.147645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:11.148509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.149024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.149535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.150455 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.152983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.156213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WARNING keystone.server.flask.common [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 30 04:46:11.176742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.sql.core [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 30 04:46:11.176742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 30 04:46:11.176742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [parameters: {'id': '93b2fa110ba3454fbee276dc2b47560f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 30 04:46:11.176742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80171) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 30 04:46:11.183736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1295e8e-425d-4bc8-8c1d-f2c39ea9c212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:11.184453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1295e8e-425d-4bc8-8c1d-f2c39ea9c212 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.184735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1295e8e-425d-4bc8-8c1d-f2c39ea9c212 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.185062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1295e8e-425d-4bc8-8c1d-f2c39ea9c212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.196427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1295e8e-425d-4bc8-8c1d-f2c39ea9c212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 93b2fa110ba3454fbee276dc2b47560f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 93b2fa110ba3454fbee276dc2b47560f. Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = self._query(query) Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application conn.query(q) Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 30 04:46:11.197208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result.read() Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 30 04:46:11.198002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application next(self.gen) Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application yield resource Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application next(self.gen) Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.session.rollback() Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.force_reraise() Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise self.value Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application session.commit() Jul 30 04:46:11.199145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._prepare_impl() Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.session.flush() Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._flush(objects) Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application compat.raise_( Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application flush_context.execute() Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 30 04:46:11.200087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rec.execute(self) Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 30 04:46:11.200998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = self._query(query) Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application conn.query(q) Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result.read() Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 30 04:46:11.201808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [parameters: {'id': '93b2fa110ba3454fbee276dc2b47560f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:11.204090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:11.205974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:11.208331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:11.208331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 30 04:46:11.208331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 30 04:46:11.208331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:11.208331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:11.208331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 30 04:46:11.208331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 30 04:46:11.208331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 93b2fa110ba3454fbee276dc2b47560f. Jul 30 04:46:11.208331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.209473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1295e8e-425d-4bc8-8c1d-f2c39ea9c212 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e1295e8e-425d-4bc8-8c1d-f2c39ea9c212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:11.216595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:11.218721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:11.221279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:11.221279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:11.221279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:11.221279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:11.221279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 476/943] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:11.221969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1295e8e-425d-4bc8-8c1d-f2c39ea9c212 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:11.222113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 467/943] 10.222.0.20 () {60 vars in 1140 bytes} [Sat Jul 30 04:46:10 2022] POST /identity/v3/projects => generated 262 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:11.224566 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:11.258832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:11.332197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1295e8e-425d-4bc8-8c1d-f2c39ea9c212 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:11.338001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646f93e-e30c-41bb-aef1-e044d90e0617 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:11.370841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505677a-8721-4387-9ca7-b4fba4341866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:11.371556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505677a-8721-4387-9ca7-b4fba4341866 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.371841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505677a-8721-4387-9ca7-b4fba4341866 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.372125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505677a-8721-4387-9ca7-b4fba4341866 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.372817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505677a-8721-4387-9ca7-b4fba4341866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:11.373123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505677a-8721-4387-9ca7-b4fba4341866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.373973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:11.374787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505677a-8721-4387-9ca7-b4fba4341866 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.381881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3dfb-30d6-42c2-9380-267e95c60729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:11.382848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3dfb-30d6-42c2-9380-267e95c60729 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.383337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3dfb-30d6-42c2-9380-267e95c60729 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.383813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3dfb-30d6-42c2-9380-267e95c60729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.393448 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3dfb-30d6-42c2-9380-267e95c60729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.393604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 477/944] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:11.394400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505677a-8721-4387-9ca7-b4fba4341866 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:11.395826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3dfb-30d6-42c2-9380-267e95c60729 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c20c3dfb-30d6-42c2-9380-267e95c60729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:46:11.409094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:11.410039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:11.410982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:46:11.410982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:46:11.410982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:46:11.410982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:46:11.410982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:46:11.410982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:46:11.410982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:11.410982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.418453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 468/945] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:11.420127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3dfb-30d6-42c2-9380-267e95c60729 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:11.494679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:11.504986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505677a-8721-4387-9ca7-b4fba4341866 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:11.531409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3dfb-30d6-42c2-9380-267e95c60729 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:11.543990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b559bf-0722-4917-86e4-691729fcd7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:11.544715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b559bf-0722-4917-86e4-691729fcd7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.545478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b559bf-0722-4917-86e4-691729fcd7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.545756 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b559bf-0722-4917-86e4-691729fcd7c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.555725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b559bf-0722-4917-86e4-691729fcd7c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.557162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b559bf-0722-4917-86e4-691729fcd7c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-54b559bf-0722-4917-86e4-691729fcd7c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:46:11.568603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:11.569623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:11.570643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:46:11.570643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:46:11.570643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:46:11.570643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:46:11.570643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:46:11.570643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:46:11.570643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:11.570643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:11.570643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 478/946] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:11.571633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35e2fa-f8b8-48f3-b509-75ab82b2ad66 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.571633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35e2fa-f8b8-48f3-b509-75ab82b2ad66 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.571885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35e2fa-f8b8-48f3-b509-75ab82b2ad66 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.573229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 479/947] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:11.577370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a1436-b43b-4854-ab21-9e3633434874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:11.578274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a1436-b43b-4854-ab21-9e3633434874 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.578504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35e2fa-f8b8-48f3-b509-75ab82b2ad66 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:11.578617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a1436-b43b-4854-ab21-9e3633434874 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.578897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a1436-b43b-4854-ab21-9e3633434874 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.579700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a1436-b43b-4854-ab21-9e3633434874 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:11.580005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a1436-b43b-4854-ab21-9e3633434874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.581483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a1436-b43b-4854-ab21-9e3633434874 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.604459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 469/948] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:46:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:11.605770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb81c8-a972-4119-8c70-dbd034f1f960 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.606049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb81c8-a972-4119-8c70-dbd034f1f960 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.606309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb81c8-a972-4119-8c70-dbd034f1f960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.635593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-eebb81c8-a972-4119-8c70-dbd034f1f960 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:11.637779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eebb81c8-a972-4119-8c70-dbd034f1f960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:11.709946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:11.722864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35e2fa-f8b8-48f3-b509-75ab82b2ad66 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:11.742578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 470/949] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:11.767368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb81c8-a972-4119-8c70-dbd034f1f960 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:11.773861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e822-46b5-4b95-9381-dd221dc225d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:11.774554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e822-46b5-4b95-9381-dd221dc225d7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.774830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e822-46b5-4b95-9381-dd221dc225d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.775103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464e822-46b5-4b95-9381-dd221dc225d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.776148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e822-46b5-4b95-9381-dd221dc225d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:11.776438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e822-46b5-4b95-9381-dd221dc225d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.778428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464e822-46b5-4b95-9381-dd221dc225d7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.791521 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 480/950] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:46:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:11.797908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e822-46b5-4b95-9381-dd221dc225d7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:11.819065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:11.891568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb81c8-a972-4119-8c70-dbd034f1f960 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:11.921314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464e822-46b5-4b95-9381-dd221dc225d7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:11.929286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c3fd0-d304-4d03-8904-af0386c55b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:11.929936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c3fd0-d304-4d03-8904-af0386c55b27 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.930211 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c3fd0-d304-4d03-8904-af0386c55b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.930512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c3fd0-d304-4d03-8904-af0386c55b27 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.931222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c3fd0-d304-4d03-8904-af0386c55b27 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.932580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c3fd0-d304-4d03-8904-af0386c55b27 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-108c3fd0-d304-4d03-8904-af0386c55b27 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:11.942044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:11.943374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:11.944592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:11.948210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 471/951] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:11.949140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c3fd0-d304-4d03-8904-af0386c55b27 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:11.958295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deeeef3-ea3a-4442-b5b2-f00b8082289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:11.959010 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deeeef3-ea3a-4442-b5b2-f00b8082289d None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:11.959295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deeeef3-ea3a-4442-b5b2-f00b8082289d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:11.959572 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deeeef3-ea3a-4442-b5b2-f00b8082289d None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/e0f2d5364cb84337b77956dc5da78b4d/roles/be2caee164e146c9ab3f7cd86636da45` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:11.961951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deeeef3-ea3a-4442-b5b2-f00b8082289d None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=e0f2d5364cb84337b77956dc5da78b4d, role_id=be2caee164e146c9ab3f7cd86636da45)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:11.963544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deeeef3-ea3a-4442-b5b2-f00b8082289d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:11.981712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 481/952] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:46:11 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/e0f2d5364cb84337b77956dc5da78b4d/roles/be2caee164e146c9ab3f7cd86636da45 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:46:12.001263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:12.063836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c3fd0-d304-4d03-8904-af0386c55b27 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:12.102005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc818-0290-4c36-bb26-0ca7e8fd24e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:12.102683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc818-0290-4c36-bb26-0ca7e8fd24e3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:12.102958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc818-0290-4c36-bb26-0ca7e8fd24e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:12.103225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc818-0290-4c36-bb26-0ca7e8fd24e3 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:12.103906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddc818-0290-4c36-bb26-0ca7e8fd24e3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:12.104203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddc818-0290-4c36-bb26-0ca7e8fd24e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:12.105652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddc818-0290-4c36-bb26-0ca7e8fd24e3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:12.111551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 472/953] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:12.117367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deeeef3-ea3a-4442-b5b2-f00b8082289d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:12.199536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:12.232528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deeeef3-ea3a-4442-b5b2-f00b8082289d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:12.269445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a1666-4218-4186-919b-7ca9af92a5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:12.270070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a1666-4218-4186-919b-7ca9af92a5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:12.270347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a1666-4218-4186-919b-7ca9af92a5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:12.270646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a1666-4218-4186-919b-7ca9af92a5c1 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:12.271317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a1666-4218-4186-919b-7ca9af92a5c1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:12.271617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a1666-4218-4186-919b-7ca9af92a5c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:12.273035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a1666-4218-4186-919b-7ca9af92a5c1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:12.279520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 482/954] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:12.285230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc818-0290-4c36-bb26-0ca7e8fd24e3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:12.390741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc818-0290-4c36-bb26-0ca7e8fd24e3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:12.402102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:12.426916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdadbc-6ca2-4090-a85e-1c4bc16d6c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:12.427532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdadbc-6ca2-4090-a85e-1c4bc16d6c1b None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:12.427808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdadbc-6ca2-4090-a85e-1c4bc16d6c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:12.428076 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdadbc-6ca2-4090-a85e-1c4bc16d6c1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:12.428794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdadbc-6ca2-4090-a85e-1c4bc16d6c1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:12.430153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdadbc-6ca2-4090-a85e-1c4bc16d6c1b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:12.452072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 473/955] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:46:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:12.547807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8f113-9d8f-4dc0-985a-46c6fbe2954e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:12.548383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8f113-9d8f-4dc0-985a-46c6fbe2954e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:12.548782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8f113-9d8f-4dc0-985a-46c6fbe2954e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:12.550601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 483/956] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:12.557948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6bd69-d2bb-4683-8043-7af65deecdbf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:12.558314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6bd69-d2bb-4683-8043-7af65deecdbf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:12.558544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6bd69-d2bb-4683-8043-7af65deecdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:12.578827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1eb6bd69-d2bb-4683-8043-7af65deecdbf None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:12.580460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1eb6bd69-d2bb-4683-8043-7af65deecdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:12.660200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 474/957] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:12.682620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8f113-9d8f-4dc0-985a-46c6fbe2954e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:12.791070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8f113-9d8f-4dc0-985a-46c6fbe2954e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:12.827704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e37a2-e525-48cd-942e-eda55f84df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:12.828354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e37a2-e525-48cd-942e-eda55f84df0c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:12.828637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e37a2-e525-48cd-942e-eda55f84df0c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:12.828913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e37a2-e525-48cd-942e-eda55f84df0c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:12.829543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e37a2-e525-48cd-942e-eda55f84df0c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:12.830954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e37a2-e525-48cd-942e-eda55f84df0c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:12.833768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-696e37a2-e525-48cd-942e-eda55f84df0c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:12.841525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:12.842706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:12.843613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 484/958] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:12.851404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6bd69-d2bb-4683-8043-7af65deecdbf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:12.950213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:12.960443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6bd69-d2bb-4683-8043-7af65deecdbf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:12.996674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07c247-9996-4dd7-9a1a-923efd9f8ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:12.997665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c247-9996-4dd7-9a1a-923efd9f8ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:12.998168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c247-9996-4dd7-9a1a-923efd9f8ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:12.998716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07c247-9996-4dd7-9a1a-923efd9f8ca7 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:12.999601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c247-9996-4dd7-9a1a-923efd9f8ca7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:13.000123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c247-9996-4dd7-9a1a-923efd9f8ca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:13.001957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07c247-9996-4dd7-9a1a-923efd9f8ca7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:13.007992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 475/959] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:46:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:13.014998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e37a2-e525-48cd-942e-eda55f84df0c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:13.125799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e37a2-e525-48cd-942e-eda55f84df0c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:13.165158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9899664-e7df-4668-be90-2ad068665ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:13.165804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9899664-e7df-4668-be90-2ad068665ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:13.166087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9899664-e7df-4668-be90-2ad068665ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:13.166358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9899664-e7df-4668-be90-2ad068665ee0 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:13.167102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9899664-e7df-4668-be90-2ad068665ee0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:13.167407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9899664-e7df-4668-be90-2ad068665ee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:13.168764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9899664-e7df-4668-be90-2ad068665ee0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:13.173771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 485/960] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:46:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:13.809945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc9959-79c3-4454-804e-7364216f5372 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:13.810677 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc9959-79c3-4454-804e-7364216f5372 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:13.811219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc9959-79c3-4454-804e-7364216f5372 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:13.813085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 476/961] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:13.818751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7eb4f-e838-470b-a705-6c105cc710f7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:13.819039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7eb4f-e838-470b-a705-6c105cc710f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:13.819319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7eb4f-e838-470b-a705-6c105cc710f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:13.838650 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-90f7eb4f-e838-470b-a705-6c105cc710f7 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:13.840689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-90f7eb4f-e838-470b-a705-6c105cc710f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:13.904988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:13.913954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 486/962] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:13.938168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc9959-79c3-4454-804e-7364216f5372 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:14.048312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc9959-79c3-4454-804e-7364216f5372 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:14.083412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f8f06-cabf-480a-b3d1-bf5fb173aed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:14.083949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f8f06-cabf-480a-b3d1-bf5fb173aed6 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.084224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f8f06-cabf-480a-b3d1-bf5fb173aed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.084488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f8f06-cabf-480a-b3d1-bf5fb173aed6 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.085159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f8f06-cabf-480a-b3d1-bf5fb173aed6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:14.086608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f8f06-cabf-480a-b3d1-bf5fb173aed6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:14.101516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.sql.core [None req-493f8f06-cabf-480a-b3d1-bf5fb173aed6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 30 04:46:14.101516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 30 04:46:14.101516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [parameters: {'user_id': 'fe5f22ae9f6b484499a4197f5de0834f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 30 04:46:14.101516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80171) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-493f8f06-cabf-480a-b3d1-bf5fb173aed6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = self._query(query) Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application conn.query(q) Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result.read() Jul 30 04:46:14.109343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 30 04:46:14.110259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application next(self.gen) Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application yield resource Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application next(self.gen) Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.session.rollback() Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.force_reraise() Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise self.value Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application session.commit() Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 30 04:46:14.111331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._prepare_impl() Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.session.flush() Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._flush(objects) Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application compat.raise_( Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application flush_context.execute() Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rec.execute(self) Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 30 04:46:14.113061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 30 04:46:14.114238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result = self._query(query) Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application conn.query(q) Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application result.read() Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 30 04:46:14.115461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fe5f22ae9f6b484499a4197f5de0834f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:14.117348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 30 04:46:14.119304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 30 04:46:14.128291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 30 04:46:14.128291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 30 04:46:14.128291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 30 04:46:14.128291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 30 04:46:14.128291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 30 04:46:14.128291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:14.128291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 477/963] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Jul 30 04:46:13 2022] POST /identity/v3/users => generated 160 bytes in 173 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:14.129235 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7eb4f-e838-470b-a705-6c105cc710f7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:14.206471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152cc4e-818e-40e6-937f-e78a20ddcf03 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.206471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152cc4e-818e-40e6-937f-e78a20ddcf03 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.206471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152cc4e-818e-40e6-937f-e78a20ddcf03 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.215628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 478/964] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:14.217176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c0a95-37db-4d13-a28f-5dbdf9506cb4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.217668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c0a95-37db-4d13-a28f-5dbdf9506cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.218179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c0a95-37db-4d13-a28f-5dbdf9506cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.231977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7eb4f-e838-470b-a705-6c105cc710f7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:14.237386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2f4c0a95-37db-4d13-a28f-5dbdf9506cb4 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:14.239250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2f4c0a95-37db-4d13-a28f-5dbdf9506cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:14.254900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:14.277547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f6fdf-612a-4180-85db-bbb6b6660444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:14.278285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6fdf-612a-4180-85db-bbb6b6660444 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.278589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6fdf-612a-4180-85db-bbb6b6660444 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.278861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f6fdf-612a-4180-85db-bbb6b6660444 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.288200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f6fdf-612a-4180-85db-bbb6b6660444 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:14.289662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f6fdf-612a-4180-85db-bbb6b6660444 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-af3f6fdf-612a-4180-85db-bbb6b6660444 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:46:14.306685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:14.308317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:14.309300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 487/965] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:46:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:14.314754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f6fdf-612a-4180-85db-bbb6b6660444 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:14.322493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 479/966] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:14.345658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c0a95-37db-4d13-a28f-5dbdf9506cb4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:14.381703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:14.432292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f6fdf-612a-4180-85db-bbb6b6660444 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:14.469444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c83b-009e-4bef-b65d-24b8fb939b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:14.470077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c83b-009e-4bef-b65d-24b8fb939b85 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.470354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c83b-009e-4bef-b65d-24b8fb939b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.470657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c83b-009e-4bef-b65d-24b8fb939b85 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.471425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8c83b-009e-4bef-b65d-24b8fb939b85 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:14.471731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8c83b-009e-4bef-b65d-24b8fb939b85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:14.473166 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8c83b-009e-4bef-b65d-24b8fb939b85 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:14.477881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c0a95-37db-4d13-a28f-5dbdf9506cb4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:14.513662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 488/967] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Jul 30 04:46:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:14.520695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53682c-eb50-4f9f-8fed-c1c2227053d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:14.521658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53682c-eb50-4f9f-8fed-c1c2227053d4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.522109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53682c-eb50-4f9f-8fed-c1c2227053d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.522546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53682c-eb50-4f9f-8fed-c1c2227053d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.536569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53682c-eb50-4f9f-8fed-c1c2227053d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ae53682c-eb50-4f9f-8fed-c1c2227053d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:14.542850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:14.543920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:14.544893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:14.547605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 480/968] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Jul 30 04:46:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:14.553954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c83b-009e-4bef-b65d-24b8fb939b85 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:14.673613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c83b-009e-4bef-b65d-24b8fb939b85 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:14.679346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50704d27-05de-4ba3-a1f9-2208fa20019d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.679623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50704d27-05de-4ba3-a1f9-2208fa20019d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.679896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50704d27-05de-4ba3-a1f9-2208fa20019d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.681384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 481/969] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:14.686868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e7f94-a85d-4fdb-8089-6ce1959960e9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.687151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e7f94-a85d-4fdb-8089-6ce1959960e9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.687428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e7f94-a85d-4fdb-8089-6ce1959960e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.690634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:14.714224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9163a-3a44-4f8f-9a07-690325feb08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:14.714867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-119e7f94-a85d-4fdb-8089-6ce1959960e9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:14.714955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9163a-3a44-4f8f-9a07-690325feb08a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.715229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9163a-3a44-4f8f-9a07-690325feb08a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.715407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9163a-3a44-4f8f-9a07-690325feb08a None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.716061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-119e7f94-a85d-4fdb-8089-6ce1959960e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:14.716161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9163a-3a44-4f8f-9a07-690325feb08a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:14.716382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9163a-3a44-4f8f-9a07-690325feb08a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:14.717768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9163a-3a44-4f8f-9a07-690325feb08a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:14.718116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:14.727273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 489/970] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Jul 30 04:46:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:14.734212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9163a-3a44-4f8f-9a07-690325feb08a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:14.800495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 482/971] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:14.823689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e7f94-a85d-4fdb-8089-6ce1959960e9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:14.857435 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9163a-3a44-4f8f-9a07-690325feb08a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:14.895615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85572578-83b6-405a-88d4-4929ff3dad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:14.896382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85572578-83b6-405a-88d4-4929ff3dad14 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.896663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85572578-83b6-405a-88d4-4929ff3dad14 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.896933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85572578-83b6-405a-88d4-4929ff3dad14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.911096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85572578-83b6-405a-88d4-4929ff3dad14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:14.912689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85572578-83b6-405a-88d4-4929ff3dad14 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-85572578-83b6-405a-88d4-4929ff3dad14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:14.929390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:14.930299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:14.931246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:14.931246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:14.931246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:14.931246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:14.931246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 490/972] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:14.935484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e7f94-a85d-4fdb-8089-6ce1959960e9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:14.936305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85572578-83b6-405a-88d4-4929ff3dad14 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:14.965569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:14.972262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d1460-e66d-42f0-914b-2daa99b2b3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:14.973166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d1460-e66d-42f0-914b-2daa99b2b3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:14.973632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d1460-e66d-42f0-914b-2daa99b2b3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:14.973973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d1460-e66d-42f0-914b-2daa99b2b3b1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:14.974629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d1460-e66d-42f0-914b-2daa99b2b3b1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:14.976115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d1460-e66d-42f0-914b-2daa99b2b3b1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-087d1460-e66d-42f0-914b-2daa99b2b3b1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:14.985698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:14.987104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:14.988065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 483/973] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:14.992687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d1460-e66d-42f0-914b-2daa99b2b3b1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:14.996214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:15.057099 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85572578-83b6-405a-88d4-4929ff3dad14 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:15.093274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9da29-bf54-4191-a4f9-15e592800783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:15.093897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9da29-bf54-4191-a4f9-15e592800783 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.094182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9da29-bf54-4191-a4f9-15e592800783 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.094465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9da29-bf54-4191-a4f9-15e592800783 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.095159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9da29-bf54-4191-a4f9-15e592800783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:15.095462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9da29-bf54-4191-a4f9-15e592800783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:15.096827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9da29-bf54-4191-a4f9-15e592800783 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:15.108370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d1460-e66d-42f0-914b-2daa99b2b3b1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:15.109234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 491/974] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:15.114511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9da29-bf54-4191-a4f9-15e592800783 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:15.144568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be05ba-1d20-4f12-92f0-918d3314d1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:15.145180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be05ba-1d20-4f12-92f0-918d3314d1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.145468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be05ba-1d20-4f12-92f0-918d3314d1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.145744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be05ba-1d20-4f12-92f0-918d3314d1c2 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.146420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be05ba-1d20-4f12-92f0-918d3314d1c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:15.146750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be05ba-1d20-4f12-92f0-918d3314d1c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:15.148126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be05ba-1d20-4f12-92f0-918d3314d1c2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:15.153695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 484/975] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:15.159198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be05ba-1d20-4f12-92f0-918d3314d1c2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:15.209595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:15.227913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9da29-bf54-4191-a4f9-15e592800783 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:15.231151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:15.262283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5924fe4-2c7b-4aa6-a3df-f2df154b161b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:15.262945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924fe4-2c7b-4aa6-a3df-f2df154b161b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.263226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924fe4-2c7b-4aa6-a3df-f2df154b161b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.263513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924fe4-2c7b-4aa6-a3df-f2df154b161b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.265477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be05ba-1d20-4f12-92f0-918d3314d1c2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:15.270419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5924fe4-2c7b-4aa6-a3df-f2df154b161b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:15.271878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5924fe4-2c7b-4aa6-a3df-f2df154b161b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a5924fe4-2c7b-4aa6-a3df-f2df154b161b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:46:15.284148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:15.285285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 30 04:46:15.286245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:15.290173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 492/976] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:46:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:15.291415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a480b2-6ddf-4220-8768-a651d8cece8b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.291699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a480b2-6ddf-4220-8768-a651d8cece8b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.292218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a480b2-6ddf-4220-8768-a651d8cece8b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.293510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 493/977] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:15.294187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a480b2-6ddf-4220-8768-a651d8cece8b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:15.300581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef6d53-9670-43d7-8bc0-b990fb39b881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:15.301195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6d53-9670-43d7-8bc0-b990fb39b881 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.301470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6d53-9670-43d7-8bc0-b990fb39b881 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.301736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6d53-9670-43d7-8bc0-b990fb39b881 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.302405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6d53-9670-43d7-8bc0-b990fb39b881 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:15.302743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6d53-9670-43d7-8bc0-b990fb39b881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:15.304090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6d53-9670-43d7-8bc0-b990fb39b881 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:15.309202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 485/978] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:15.310267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c921e7-cf16-4dfb-86d3-a50f101263c7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.310580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c921e7-cf16-4dfb-86d3-a50f101263c7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.310850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c921e7-cf16-4dfb-86d3-a50f101263c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.328932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-93c921e7-cf16-4dfb-86d3-a50f101263c7 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:15.330141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-93c921e7-cf16-4dfb-86d3-a50f101263c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:15.394648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 486/979] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:15.395567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c921e7-cf16-4dfb-86d3-a50f101263c7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:15.399499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a480b2-6ddf-4220-8768-a651d8cece8b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:15.434581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eebe8c-e80d-4ca3-9c20-13541c2ed21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:15.435195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eebe8c-e80d-4ca3-9c20-13541c2ed21c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.435471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eebe8c-e80d-4ca3-9c20-13541c2ed21c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.435740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eebe8c-e80d-4ca3-9c20-13541c2ed21c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.436534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eebe8c-e80d-4ca3-9c20-13541c2ed21c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:15.436818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eebe8c-e80d-4ca3-9c20-13541c2ed21c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:15.438247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eebe8c-e80d-4ca3-9c20-13541c2ed21c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:15.455908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 494/980] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Jul 30 04:46:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:15.456692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eebe8c-e80d-4ca3-9c20-13541c2ed21c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:15.457723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:15.462192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:15.497953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c921e7-cf16-4dfb-86d3-a50f101263c7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:15.531202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd18b3-7eb5-4d65-952c-04fe8b049b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:15.531810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd18b3-7eb5-4d65-952c-04fe8b049b19 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.532086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd18b3-7eb5-4d65-952c-04fe8b049b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.532355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd18b3-7eb5-4d65-952c-04fe8b049b19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.533106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd18b3-7eb5-4d65-952c-04fe8b049b19 None admin] RBAC: Adding query filter params (interface=admin, service_id=82ebd465bd2f42469f086d3dc44c0dd1, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:15.533408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd18b3-7eb5-4d65-952c-04fe8b049b19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:15.534792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd18b3-7eb5-4d65-952c-04fe8b049b19 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:15.539723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 487/981] 10.222.0.20 () {56 vars in 1249 bytes} [Sat Jul 30 04:46:15 2022] GET /identity/v3/endpoints?service_id=82ebd465bd2f42469f086d3dc44c0dd1&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:15.540457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd18b3-7eb5-4d65-952c-04fe8b049b19 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:15.563184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eebe8c-e80d-4ca3-9c20-13541c2ed21c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:15.597149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53927031-413d-4fc7-ac8a-1c414af45ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:15.597762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53927031-413d-4fc7-ac8a-1c414af45ada None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.598028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53927031-413d-4fc7-ac8a-1c414af45ada None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.598305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53927031-413d-4fc7-ac8a-1c414af45ada None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.599011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53927031-413d-4fc7-ac8a-1c414af45ada None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:15.600366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53927031-413d-4fc7-ac8a-1c414af45ada None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:15.615423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 495/982] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:46:15 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:15.616228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53927031-413d-4fc7-ac8a-1c414af45ada None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:15.648099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd18b3-7eb5-4d65-952c-04fe8b049b19 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:15.679199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:15.694351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:15.703517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbfc07-060d-47b7-ac3b-e5d4fd9b90f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:15.704209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbfc07-060d-47b7-ac3b-e5d4fd9b90f7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.704485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbfc07-060d-47b7-ac3b-e5d4fd9b90f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.704759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbfc07-060d-47b7-ac3b-e5d4fd9b90f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.744097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53927031-413d-4fc7-ac8a-1c414af45ada None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:15.785693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbfc07-060d-47b7-ac3b-e5d4fd9b90f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:15.786192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2cbd7-7ccb-45a2-a285-40eaca535ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:15.786576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbfc07-060d-47b7-ac3b-e5d4fd9b90f7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:15.786853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2cbd7-7ccb-45a2-a285-40eaca535ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:15.788141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2cbd7-7ccb-45a2-a285-40eaca535ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:15.788141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2cbd7-7ccb-45a2-a285-40eaca535ad1 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:15.788141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2cbd7-7ccb-45a2-a285-40eaca535ad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:15.789530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2cbd7-7ccb-45a2-a285-40eaca535ad1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:15.796632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 496/983] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Jul 30 04:46:15 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:46:15.905468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 488/984] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:15 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:15.906881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2cbd7-7ccb-45a2-a285-40eaca535ad1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:16.028476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2cbd7-7ccb-45a2-a285-40eaca535ad1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:16.056758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:16.066271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f919d4a-4f19-4891-a5c7-ef691c3c3483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:16.066951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f919d4a-4f19-4891-a5c7-ef691c3c3483 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:16.067241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f919d4a-4f19-4891-a5c7-ef691c3c3483 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:16.067519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f919d4a-4f19-4891-a5c7-ef691c3c3483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:16.079108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f919d4a-4f19-4891-a5c7-ef691c3c3483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:16.080563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f919d4a-4f19-4891-a5c7-ef691c3c3483 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8f919d4a-4f19-4891-a5c7-ef691c3c3483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:16.095496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:16.096429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:16.097439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:16.097439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:16.097439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:16.097439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:16.097439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 497/985] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:16.102566 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbfc07-060d-47b7-ac3b-e5d4fd9b90f7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:16.127294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:16.209022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbfc07-060d-47b7-ac3b-e5d4fd9b90f7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:16.244272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170e54-fb2d-4e80-a233-d042dff5ddb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:16.247659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170e54-fb2d-4e80-a233-d042dff5ddb6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:16.247659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170e54-fb2d-4e80-a233-d042dff5ddb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:16.247659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170e54-fb2d-4e80-a233-d042dff5ddb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:16.247659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15170e54-fb2d-4e80-a233-d042dff5ddb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:16.247659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15170e54-fb2d-4e80-a233-d042dff5ddb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:16.248354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15170e54-fb2d-4e80-a233-d042dff5ddb6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:16.266298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 489/986] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:16.267587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f919d4a-4f19-4891-a5c7-ef691c3c3483 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:16.374293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f919d4a-4f19-4891-a5c7-ef691c3c3483 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:16.408743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4d8fe-d9ab-4608-b2ee-81d55a877a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:16.409382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d8fe-d9ab-4608-b2ee-81d55a877a34 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:16.409659 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d8fe-d9ab-4608-b2ee-81d55a877a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:16.409927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d8fe-d9ab-4608-b2ee-81d55a877a34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:16.416881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4d8fe-d9ab-4608-b2ee-81d55a877a34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:16.418325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4d8fe-d9ab-4608-b2ee-81d55a877a34 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-88f4d8fe-d9ab-4608-b2ee-81d55a877a34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:46:16.429993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:16.431203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:16.432109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:46:16.432109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:46:16.432109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:46:16.432109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:46:16.432109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:46:16.432109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:46:16.432109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:16.432109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:16.432109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 498/987] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:16.440308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170e54-fb2d-4e80-a233-d042dff5ddb6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:16.526791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:16.554919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170e54-fb2d-4e80-a233-d042dff5ddb6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:16.596781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a8580-8cd4-4a7c-bebd-3e7137f63378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:16.597402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a8580-8cd4-4a7c-bebd-3e7137f63378 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:16.597681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a8580-8cd4-4a7c-bebd-3e7137f63378 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:16.598014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a8580-8cd4-4a7c-bebd-3e7137f63378 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:16.599780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a8580-8cd4-4a7c-bebd-3e7137f63378 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:16.600112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a8580-8cd4-4a7c-bebd-3e7137f63378 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:16.601551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a8580-8cd4-4a7c-bebd-3e7137f63378 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:16.619777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 490/988] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:16.626960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4d8fe-d9ab-4608-b2ee-81d55a877a34 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:16.692935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:16.777228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4d8fe-d9ab-4608-b2ee-81d55a877a34 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:16.814372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f3631-597b-40dc-b268-0c4d6bf5c8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:16.815037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f3631-597b-40dc-b268-0c4d6bf5c8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:16.815315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f3631-597b-40dc-b268-0c4d6bf5c8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:16.815585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f3631-597b-40dc-b268-0c4d6bf5c8d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:16.816374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f3631-597b-40dc-b268-0c4d6bf5c8d3 None admin] RBAC: Adding query filter params (role.id=be2caee164e146c9ab3f7cd86636da45, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=e0f2d5364cb84337b77956dc5da78b4d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:16.816679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f3631-597b-40dc-b268-0c4d6bf5c8d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:16.818118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f3631-597b-40dc-b268-0c4d6bf5c8d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:16.825273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 499/989] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:16 2022] GET /identity/v3/role_assignments?user.id=e0f2d5364cb84337b77956dc5da78b4d&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=be2caee164e146c9ab3f7cd86636da45 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:17.704077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0bbf0-1c31-4b65-ab7f-c6c3129457eb None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:17.704636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0bbf0-1c31-4b65-ab7f-c6c3129457eb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:17.704636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0bbf0-1c31-4b65-ab7f-c6c3129457eb None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:17.706108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 491/990] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:17.710989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e9a24-f044-416f-b485-5aad2d7ea1c9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:17.711395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e9a24-f044-416f-b485-5aad2d7ea1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:17.711563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e9a24-f044-416f-b485-5aad2d7ea1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:17.730819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cd3e9a24-f044-416f-b485-5aad2d7ea1c9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:17.731998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd3e9a24-f044-416f-b485-5aad2d7ea1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:17.802596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 500/991] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:17.822049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0bbf0-1c31-4b65-ab7f-c6c3129457eb None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:17.860351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aee02c-5fe1-4bcf-b7ed-eef9c9b8eb54 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:17.860642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aee02c-5fe1-4bcf-b7ed-eef9c9b8eb54 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:17.860923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aee02c-5fe1-4bcf-b7ed-eef9c9b8eb54 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:17.862289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 501/992] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:17.867139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2c6a4-9158-4d08-9d29-e00f6f58009b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:17.867425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2c6a4-9158-4d08-9d29-e00f6f58009b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:17.867699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2c6a4-9158-4d08-9d29-e00f6f58009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:17.886677 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3fc2c6a4-9158-4d08-9d29-e00f6f58009b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:17.887956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3fc2c6a4-9158-4d08-9d29-e00f6f58009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:17.916604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:17.931916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0bbf0-1c31-4b65-ab7f-c6c3129457eb None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:17.959959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 502/993] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:17.967197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd961639-6b8a-4986-a81c-eaa4602cff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:17.967842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd961639-6b8a-4986-a81c-eaa4602cff09 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:17.968117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd961639-6b8a-4986-a81c-eaa4602cff09 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:17.968390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd961639-6b8a-4986-a81c-eaa4602cff09 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:17.969010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd961639-6b8a-4986-a81c-eaa4602cff09 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:17.970400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd961639-6b8a-4986-a81c-eaa4602cff09 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-dd961639-6b8a-4986-a81c-eaa4602cff09 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:17.979940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:17.981121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:17.982003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 492/994] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:17.985241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2c6a4-9158-4d08-9d29-e00f6f58009b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:17.986862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd961639-6b8a-4986-a81c-eaa4602cff09 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:17.990649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:18.095985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd961639-6b8a-4986-a81c-eaa4602cff09 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:18.104900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2c6a4-9158-4d08-9d29-e00f6f58009b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:18.129651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923be1c-966c-4d58-8d94-f961351b2a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:18.130268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923be1c-966c-4d58-8d94-f961351b2a6b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.130579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923be1c-966c-4d58-8d94-f961351b2a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.130851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c923be1c-966c-4d58-8d94-f961351b2a6b None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.131518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923be1c-966c-4d58-8d94-f961351b2a6b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:18.131822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923be1c-966c-4d58-8d94-f961351b2a6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:18.133195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c923be1c-966c-4d58-8d94-f961351b2a6b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:18.138573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 493/995] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:46:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:18.143928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcbe6f-b279-462e-bdca-e5f32b2de786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:18.144161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923be1c-966c-4d58-8d94-f961351b2a6b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:18.144888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcbe6f-b279-462e-bdca-e5f32b2de786 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.145390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcbe6f-b279-462e-bdca-e5f32b2de786 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.145804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcbe6f-b279-462e-bdca-e5f32b2de786 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.146722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcbe6f-b279-462e-bdca-e5f32b2de786 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:18.148342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcbe6f-b279-462e-bdca-e5f32b2de786 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fadcbe6f-b279-462e-bdca-e5f32b2de786 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:18.158685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:18.159888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 30 04:46:18.160925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:18.163343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 503/996] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Jul 30 04:46:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:18.169380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcbe6f-b279-462e-bdca-e5f32b2de786 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:18.203265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:18.217646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:18.253646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c923be1c-966c-4d58-8d94-f961351b2a6b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:18.280735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcbe6f-b279-462e-bdca-e5f32b2de786 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:18.291529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a53e3-7101-44eb-adb5-ec2f5b44d1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:18.292724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a53e3-7101-44eb-adb5-ec2f5b44d1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.292999 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a53e3-7101-44eb-adb5-ec2f5b44d1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.293266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a53e3-7101-44eb-adb5-ec2f5b44d1a3 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.293935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a53e3-7101-44eb-adb5-ec2f5b44d1a3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:18.294241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a53e3-7101-44eb-adb5-ec2f5b44d1a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:18.295640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a53e3-7101-44eb-adb5-ec2f5b44d1a3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:18.300516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 494/997] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:46:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:18.305947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a53e3-7101-44eb-adb5-ec2f5b44d1a3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:18.320273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be8805-74fe-4b61-a71b-d44274673862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:18.321179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be8805-74fe-4b61-a71b-d44274673862 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.321640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be8805-74fe-4b61-a71b-d44274673862 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.322097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be8805-74fe-4b61-a71b-d44274673862 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.322808 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be8805-74fe-4b61-a71b-d44274673862 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:18.323649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be8805-74fe-4b61-a71b-d44274673862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:18.325029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be8805-74fe-4b61-a71b-d44274673862 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:18.334492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 504/998] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:18.336183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be8805-74fe-4b61-a71b-d44274673862 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:18.412767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a53e3-7101-44eb-adb5-ec2f5b44d1a3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:18.438367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:18.443149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:18.448985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baff12d-52ac-4d0f-9ca2-c3c61cc8b8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:18.449600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baff12d-52ac-4d0f-9ca2-c3c61cc8b8ed None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.449876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baff12d-52ac-4d0f-9ca2-c3c61cc8b8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.450143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baff12d-52ac-4d0f-9ca2-c3c61cc8b8ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.450910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baff12d-52ac-4d0f-9ca2-c3c61cc8b8ed None admin] RBAC: Adding query filter params (interface=public, service_id=990039041463485a9e748822ab9bcffd, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:18.451154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be8805-74fe-4b61-a71b-d44274673862 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:18.451254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baff12d-52ac-4d0f-9ca2-c3c61cc8b8ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:18.452567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baff12d-52ac-4d0f-9ca2-c3c61cc8b8ed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:18.457979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 495/999] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Jul 30 04:46:18 2022] GET /identity/v3/endpoints?service_id=990039041463485a9e748822ab9bcffd&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:18.463249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baff12d-52ac-4d0f-9ca2-c3c61cc8b8ed None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:18.490390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbbd14-e44d-46dd-91e5-af710026cb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:18.491626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbbd14-e44d-46dd-91e5-af710026cb75 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.491911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbbd14-e44d-46dd-91e5-af710026cb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.492188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbbd14-e44d-46dd-91e5-af710026cb75 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.492865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbbd14-e44d-46dd-91e5-af710026cb75 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:18.493171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbbd14-e44d-46dd-91e5-af710026cb75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:18.494910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfbbd14-e44d-46dd-91e5-af710026cb75 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:18.500745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 505/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Jul 30 04:46:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:18.506199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbbd14-e44d-46dd-91e5-af710026cb75 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:18.566298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baff12d-52ac-4d0f-9ca2-c3c61cc8b8ed None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:18.601965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b9ce6-188a-4f59-a7a8-50167e267ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:18.604798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b9ce6-188a-4f59-a7a8-50167e267ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.605357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b9ce6-188a-4f59-a7a8-50167e267ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.605800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b9ce6-188a-4f59-a7a8-50167e267ea2 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.606486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b9ce6-188a-4f59-a7a8-50167e267ea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:18.607878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b9ce6-188a-4f59-a7a8-50167e267ea2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:18.622465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 496/1001] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Jul 30 04:46:18 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:46:18.627214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbbd14-e44d-46dd-91e5-af710026cb75 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:18.665764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f582e58-1497-487c-af3d-45926272d582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:18.666408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f582e58-1497-487c-af3d-45926272d582 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.667273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f582e58-1497-487c-af3d-45926272d582 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.667558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f582e58-1497-487c-af3d-45926272d582 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.668376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f582e58-1497-487c-af3d-45926272d582 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:18.669933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f582e58-1497-487c-af3d-45926272d582 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:18.694470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 506/1002] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:46:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:46:18.930253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509493cb-58e1-4a24-813e-6cce8585b239 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.930789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509493cb-58e1-4a24-813e-6cce8585b239 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.930789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509493cb-58e1-4a24-813e-6cce8585b239 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.932284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 497/1003] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:18.937351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a4907-8830-4d28-b408-fc0758bf9b43 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:18.937763 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a4907-8830-4d28-b408-fc0758bf9b43 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:18.937919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a4907-8830-4d28-b408-fc0758bf9b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:18.956873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b60a4907-8830-4d28-b408-fc0758bf9b43 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:18.958074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b60a4907-8830-4d28-b408-fc0758bf9b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:18.969891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:19.042186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 507/1004] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:18 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:19.064031 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509493cb-58e1-4a24-813e-6cce8585b239 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:19.125693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:19.169691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509493cb-58e1-4a24-813e-6cce8585b239 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:19.205701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f5644-a177-4ef7-af7d-17e91162eb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:19.206346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f5644-a177-4ef7-af7d-17e91162eb14 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:19.206655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f5644-a177-4ef7-af7d-17e91162eb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:19.206930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f5644-a177-4ef7-af7d-17e91162eb14 None admin] PATH_INFO: `/v3/services/image` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:19.207582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f5644-a177-4ef7-af7d-17e91162eb14 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:19.208943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f5644-a177-4ef7-af7d-17e91162eb14 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7c7f5644-a177-4ef7-af7d-17e91162eb14 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:19.218776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:19.219774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 30 04:46:19.220664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:19.227878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 498/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Sat Jul 30 04:46:19 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:19.228019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a4907-8830-4d28-b408-fc0758bf9b43 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:19.326980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a4907-8830-4d28-b408-fc0758bf9b43 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:19.362294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786529e-af58-432c-acd2-43678e14f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:19.362965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786529e-af58-432c-acd2-43678e14f912 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:19.363250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786529e-af58-432c-acd2-43678e14f912 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:19.363528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786529e-af58-432c-acd2-43678e14f912 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:19.364207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786529e-af58-432c-acd2-43678e14f912 None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:19.364512 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786529e-af58-432c-acd2-43678e14f912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:19.365885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786529e-af58-432c-acd2-43678e14f912 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:19.375648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 508/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:19.376675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f5644-a177-4ef7-af7d-17e91162eb14 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:19.478247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f5644-a177-4ef7-af7d-17e91162eb14 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:19.503200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:19.515536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2fffe-af58-4ff1-a521-f4acedc1edf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:19.516190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2fffe-af58-4ff1-a521-f4acedc1edf8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:19.516433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2fffe-af58-4ff1-a521-f4acedc1edf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:19.516706 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2fffe-af58-4ff1-a521-f4acedc1edf8 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:19.517384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2fffe-af58-4ff1-a521-f4acedc1edf8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:19.517697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2fffe-af58-4ff1-a521-f4acedc1edf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:19.519098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2fffe-af58-4ff1-a521-f4acedc1edf8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:19.529290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 499/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:19 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:20.650603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e6755-2371-48f3-a300-29b66e0dcef1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:20.651804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e6755-2371-48f3-a300-29b66e0dcef1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:20.651804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e6755-2371-48f3-a300-29b66e0dcef1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:20.653481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 509/1008] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:20.658548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99309385-15e2-4330-8e44-0961da4af4d9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:20.658943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99309385-15e2-4330-8e44-0961da4af4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:20.659141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99309385-15e2-4330-8e44-0961da4af4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:20.677776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-99309385-15e2-4330-8e44-0961da4af4d9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:20.679000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-99309385-15e2-4330-8e44-0961da4af4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:20.749221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 500/1009] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:20.771356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e6755-2371-48f3-a300-29b66e0dcef1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:20.781308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:20.800216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec94a-a02c-473c-9b24-f18032276944 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:20.800505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec94a-a02c-473c-9b24-f18032276944 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:20.800776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ec94a-a02c-473c-9b24-f18032276944 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:20.802277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 501/1010] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:20.806922 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb396c2-d92a-4fc9-99a2-1a15fd6027d0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:20.807357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb396c2-d92a-4fc9-99a2-1a15fd6027d0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:20.808112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb396c2-d92a-4fc9-99a2-1a15fd6027d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:20.826328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4bb396c2-d92a-4fc9-99a2-1a15fd6027d0 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:20.827545 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4bb396c2-d92a-4fc9-99a2-1a15fd6027d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:20.886685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e6755-2371-48f3-a300-29b66e0dcef1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:20.899924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 502/1011] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:20.923190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb396c2-d92a-4fc9-99a2-1a15fd6027d0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:20.924437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9d806-e081-4182-a198-dc8e22442c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:20.925087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9d806-e081-4182-a198-dc8e22442c14 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:20.925367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9d806-e081-4182-a198-dc8e22442c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:20.925644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9d806-e081-4182-a198-dc8e22442c14 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:20.926268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9d806-e081-4182-a198-dc8e22442c14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:20.927661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9d806-e081-4182-a198-dc8e22442c14 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1fb9d806-e081-4182-a198-dc8e22442c14 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:20.938019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:20.939168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:20.940113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 510/1012] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:20.944961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9d806-e081-4182-a198-dc8e22442c14 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:20.986642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:21.031285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:21.055306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb396c2-d92a-4fc9-99a2-1a15fd6027d0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:21.067451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9d806-e081-4182-a198-dc8e22442c14 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:21.091689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc948d-e206-447b-9126-9f74ca479893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:21.092325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc948d-e206-447b-9126-9f74ca479893 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:21.092597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc948d-e206-447b-9126-9f74ca479893 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:21.092867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc948d-e206-447b-9126-9f74ca479893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:21.093996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc948d-e206-447b-9126-9f74ca479893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:21.095454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc948d-e206-447b-9126-9f74ca479893 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:21.097037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 503/1013] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:21.102632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc948d-e206-447b-9126-9f74ca479893 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:21.107069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b99335-6f03-4b78-ab79-d394e1393368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:21.107693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b99335-6f03-4b78-ab79-d394e1393368 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:21.107975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b99335-6f03-4b78-ab79-d394e1393368 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:21.108243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b99335-6f03-4b78-ab79-d394e1393368 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:21.108918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b99335-6f03-4b78-ab79-d394e1393368 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:21.109219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b99335-6f03-4b78-ab79-d394e1393368 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:21.110600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b99335-6f03-4b78-ab79-d394e1393368 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:21.115678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 511/1014] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:46:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:21.122055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b99335-6f03-4b78-ab79-d394e1393368 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:21.215447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc948d-e206-447b-9126-9f74ca479893 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:21.227319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b99335-6f03-4b78-ab79-d394e1393368 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:21.232812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:21.252020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00996e-85db-4bab-8010-500a8049f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:21.252639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00996e-85db-4bab-8010-500a8049f8f0 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:21.252918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00996e-85db-4bab-8010-500a8049f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:21.253198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00996e-85db-4bab-8010-500a8049f8f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:21.253875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00996e-85db-4bab-8010-500a8049f8f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:21.255344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00996e-85db-4bab-8010-500a8049f8f0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:21.264407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e97e-a0eb-41b6-998a-f88f165ed29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:21.265030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e97e-a0eb-41b6-998a-f88f165ed29c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:21.265309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e97e-a0eb-41b6-998a-f88f165ed29c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:21.265579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e97e-a0eb-41b6-998a-f88f165ed29c None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:21.266734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5e97e-a0eb-41b6-998a-f88f165ed29c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:21.267038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5e97e-a0eb-41b6-998a-f88f165ed29c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:21.268800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5e97e-a0eb-41b6-998a-f88f165ed29c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:21.270100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:21.274795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 512/1015] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Jul 30 04:46:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:21.281553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e97e-a0eb-41b6-998a-f88f165ed29c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:21.298465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 504/1016] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:46:21 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:46:21.408813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e97e-a0eb-41b6-998a-f88f165ed29c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:21.446034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e004ec5-9bab-48d7-ae6e-860efa400b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:21.446691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e004ec5-9bab-48d7-ae6e-860efa400b58 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:21.446976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e004ec5-9bab-48d7-ae6e-860efa400b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:21.447242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e004ec5-9bab-48d7-ae6e-860efa400b58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:21.448486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e004ec5-9bab-48d7-ae6e-860efa400b58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:21.449858 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e004ec5-9bab-48d7-ae6e-860efa400b58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:21.472172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 513/1017] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:46:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:46:21.537978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d1e0-406b-450f-9a4c-3d93d3759f28 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:21.538314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d1e0-406b-450f-9a4c-3d93d3759f28 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:21.538572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d1e0-406b-450f-9a4c-3d93d3759f28 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:21.540142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 505/1018] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:21.545700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad6f63-6ac5-4c53-878f-c2fe9afe7866 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:21.546003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad6f63-6ac5-4c53-878f-c2fe9afe7866 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:21.546265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad6f63-6ac5-4c53-878f-c2fe9afe7866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:21.566789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d1ad6f63-6ac5-4c53-878f-c2fe9afe7866 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:21.568016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1ad6f63-6ac5-4c53-878f-c2fe9afe7866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:21.627027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:21.658586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 514/1019] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:21 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:21.677795 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86d1e0-406b-450f-9a4c-3d93d3759f28 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:21.793525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86d1e0-406b-450f-9a4c-3d93d3759f28 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:21.829226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f891c-8823-48dc-9509-845aa664cad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:21.829849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f891c-8823-48dc-9509-845aa664cad8 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:21.830124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f891c-8823-48dc-9509-845aa664cad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:21.830394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f891c-8823-48dc-9509-845aa664cad8 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:21.831140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f891c-8823-48dc-9509-845aa664cad8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:21.832500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f891c-8823-48dc-9509-845aa664cad8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:21.848625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 506/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:46:21 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:23.393806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a5a85-378f-4f4d-af75-8e621b03869e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:23.394314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a5a85-378f-4f4d-af75-8e621b03869e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:23.394314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a5a85-378f-4f4d-af75-8e621b03869e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:23.395695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 515/1021] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:23.401068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d543fb-7091-4956-9ef1-6144708a2f69 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:23.401484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d543fb-7091-4956-9ef1-6144708a2f69 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:23.401667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d543fb-7091-4956-9ef1-6144708a2f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:23.420294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-19d543fb-7091-4956-9ef1-6144708a2f69 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:23.421499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-19d543fb-7091-4956-9ef1-6144708a2f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:23.445867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:23.500847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 507/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:23.525024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a5a85-378f-4f4d-af75-8e621b03869e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:23.625103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddadf03-e000-41f5-87c8-8143716924ec None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:23.625494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddadf03-e000-41f5-87c8-8143716924ec None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:23.625834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddadf03-e000-41f5-87c8-8143716924ec None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:23.627293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 508/1023] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:23.632082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a5a85-378f-4f4d-af75-8e621b03869e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:23.632218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13afaa-58a4-4ab4-9a1c-6f6b83cf1761 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:23.632494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13afaa-58a4-4ab4-9a1c-6f6b83cf1761 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:23.632774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13afaa-58a4-4ab4-9a1c-6f6b83cf1761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:23.652750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bd13afaa-58a4-4ab4-9a1c-6f6b83cf1761 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:23.653867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bd13afaa-58a4-4ab4-9a1c-6f6b83cf1761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:23.669161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50343f11-cc3a-424f-93bb-783691ffefd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:23.669808 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50343f11-cc3a-424f-93bb-783691ffefd6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:23.670091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50343f11-cc3a-424f-93bb-783691ffefd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:23.670365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50343f11-cc3a-424f-93bb-783691ffefd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:23.675705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50343f11-cc3a-424f-93bb-783691ffefd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:23.677149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50343f11-cc3a-424f-93bb-783691ffefd6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:23.678939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 516/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:23.691965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50343f11-cc3a-424f-93bb-783691ffefd6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:23.730508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 509/1025] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:23.749857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13afaa-58a4-4ab4-9a1c-6f6b83cf1761 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:23.751890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:23.806694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50343f11-cc3a-424f-93bb-783691ffefd6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:23.846715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d05076-63fd-4930-9cb4-a7d379d626bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:23.847343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d05076-63fd-4930-9cb4-a7d379d626bf None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:23.847786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d05076-63fd-4930-9cb4-a7d379d626bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:23.848014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:23.848125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d05076-63fd-4930-9cb4-a7d379d626bf None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:23.848990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d05076-63fd-4930-9cb4-a7d379d626bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:23.850954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d05076-63fd-4930-9cb4-a7d379d626bf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:23.864860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13afaa-58a4-4ab4-9a1c-6f6b83cf1761 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:23.883822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 517/1026] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:46:23 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:23.883822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bf190-8f26-49d8-bbfd-dfabe5dc8c48 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:23.883822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bf190-8f26-49d8-bbfd-dfabe5dc8c48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:23.883822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bf190-8f26-49d8-bbfd-dfabe5dc8c48 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:23.883822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 518/1027] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:23.886476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1269a-3acc-4e6f-a6a1-d726c951f397 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:23.886752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1269a-3acc-4e6f-a6a1-d726c951f397 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:23.887024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1269a-3acc-4e6f-a6a1-d726c951f397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:23.908170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dfc1269a-3acc-4e6f-a6a1-d726c951f397 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:23.909320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dfc1269a-3acc-4e6f-a6a1-d726c951f397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:23.916390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8c5d1-ab78-462e-a85a-af755bbb3016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:23.917556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8c5d1-ab78-462e-a85a-af755bbb3016 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:23.917834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8c5d1-ab78-462e-a85a-af755bbb3016 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:23.918202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8c5d1-ab78-462e-a85a-af755bbb3016 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:23.931452 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8c5d1-ab78-462e-a85a-af755bbb3016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-14c8c5d1-ab78-462e-a85a-af755bbb3016 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:23.938000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:23.939271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:23.940204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 510/1028] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Jul 30 04:46:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:23.945350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8c5d1-ab78-462e-a85a-af755bbb3016 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:23.998608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 519/1029] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:24.020542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1269a-3acc-4e6f-a6a1-d726c951f397 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.062573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:24.069869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8c5d1-ab78-462e-a85a-af755bbb3016 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.119045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54366598-1ac5-4f98-b4a8-850b54e39dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.119687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54366598-1ac5-4f98-b4a8-850b54e39dde None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.119973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54366598-1ac5-4f98-b4a8-850b54e39dde None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.120370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54366598-1ac5-4f98-b4a8-850b54e39dde None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.121107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54366598-1ac5-4f98-b4a8-850b54e39dde None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:24.121417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54366598-1ac5-4f98-b4a8-850b54e39dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.122818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54366598-1ac5-4f98-b4a8-850b54e39dde None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.134469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 511/1030] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Jul 30 04:46:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:24.138049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54366598-1ac5-4f98-b4a8-850b54e39dde None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.144177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1269a-3acc-4e6f-a6a1-d726c951f397 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.148598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:24.182502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b9160-85e6-4d48-a35e-9706fcfb1cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.183091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b9160-85e6-4d48-a35e-9706fcfb1cbe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.183462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b9160-85e6-4d48-a35e-9706fcfb1cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.183741 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b9160-85e6-4d48-a35e-9706fcfb1cbe None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.184375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b9160-85e6-4d48-a35e-9706fcfb1cbe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.185762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b9160-85e6-4d48-a35e-9706fcfb1cbe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1a9b9160-85e6-4d48-a35e-9706fcfb1cbe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:24.194991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:24.196015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:24.196904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 520/1031] 10.222.0.20 () {56 vars in 1105 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:24.203050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b9160-85e6-4d48-a35e-9706fcfb1cbe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.251395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54366598-1ac5-4f98-b4a8-850b54e39dde None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.289408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a6d3e-1d4f-46e7-b826-50169f7f8ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.290054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a6d3e-1d4f-46e7-b826-50169f7f8ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.290328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a6d3e-1d4f-46e7-b826-50169f7f8ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.290629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a6d3e-1d4f-46e7-b826-50169f7f8ef4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.302142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a6d3e-1d4f-46e7-b826-50169f7f8ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.303636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a6d3e-1d4f-46e7-b826-50169f7f8ef4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.308082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:24.312333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b9160-85e6-4d48-a35e-9706fcfb1cbe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-118a6d3e-1d4f-46e7-b826-50169f7f8ef4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:24.319702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:24.320810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:24.321909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:24.321909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:24.321909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:24.321909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:24.321909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 512/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:24.327372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a6d3e-1d4f-46e7-b826-50169f7f8ef4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.356381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc0720-b989-4d77-84bb-3df048609867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.357011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc0720-b989-4d77-84bb-3df048609867 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.357378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc0720-b989-4d77-84bb-3df048609867 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.357625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc0720-b989-4d77-84bb-3df048609867 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.358239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc0720-b989-4d77-84bb-3df048609867 None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:24.358573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc0720-b989-4d77-84bb-3df048609867 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.359950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc0720-b989-4d77-84bb-3df048609867 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.365135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 521/1033] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:24.371019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc0720-b989-4d77-84bb-3df048609867 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.392426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:24.445984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a6d3e-1d4f-46e7-b826-50169f7f8ef4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.480114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc0720-b989-4d77-84bb-3df048609867 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.485471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274b17c-aac0-4a6c-b4e9-4300d05ac559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.486092 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274b17c-aac0-4a6c-b4e9-4300d05ac559 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.486371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274b17c-aac0-4a6c-b4e9-4300d05ac559 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.486669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274b17c-aac0-4a6c-b4e9-4300d05ac559 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.487656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274b17c-aac0-4a6c-b4e9-4300d05ac559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:24.487765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274b17c-aac0-4a6c-b4e9-4300d05ac559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.489062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274b17c-aac0-4a6c-b4e9-4300d05ac559 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.502715 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 513/1034] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:24.508032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274b17c-aac0-4a6c-b4e9-4300d05ac559 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.516272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f569073-3715-41b7-8aee-21ae86797859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.516905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f569073-3715-41b7-8aee-21ae86797859 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.517185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f569073-3715-41b7-8aee-21ae86797859 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.517462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f569073-3715-41b7-8aee-21ae86797859 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.518141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f569073-3715-41b7-8aee-21ae86797859 None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:24.518468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f569073-3715-41b7-8aee-21ae86797859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.519840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f569073-3715-41b7-8aee-21ae86797859 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.524784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 522/1035] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:24.531276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f569073-3715-41b7-8aee-21ae86797859 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.560395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:24.614696 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274b17c-aac0-4a6c-b4e9-4300d05ac559 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.634044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:24.643824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f569073-3715-41b7-8aee-21ae86797859 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.655511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc983ccc-2046-4dd6-9b98-91977a7ecc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.655796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc983ccc-2046-4dd6-9b98-91977a7ecc9b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.655940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc983ccc-2046-4dd6-9b98-91977a7ecc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.656192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc983ccc-2046-4dd6-9b98-91977a7ecc9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.663760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc983ccc-2046-4dd6-9b98-91977a7ecc9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.665184 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc983ccc-2046-4dd6-9b98-91977a7ecc9b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-bc983ccc-2046-4dd6-9b98-91977a7ecc9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:46:24.677262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:24.678265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:24.679275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 514/1036] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:24.684891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc983ccc-2046-4dd6-9b98-91977a7ecc9b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.685808 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62106b-fc12-4a8f-b830-caac729896de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.686480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62106b-fc12-4a8f-b830-caac729896de None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.686796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62106b-fc12-4a8f-b830-caac729896de None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.687073 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62106b-fc12-4a8f-b830-caac729896de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.687837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62106b-fc12-4a8f-b830-caac729896de None admin] RBAC: Adding query filter params (interface=public, service_id=9307fcbc4833454e9a5f5ca89979b751, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:24.688148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62106b-fc12-4a8f-b830-caac729896de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.689528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a62106b-fc12-4a8f-b830-caac729896de None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.695091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 523/1037] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/endpoints?service_id=9307fcbc4833454e9a5f5ca89979b751&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:24.702539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62106b-fc12-4a8f-b830-caac729896de None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.792024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc983ccc-2046-4dd6-9b98-91977a7ecc9b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.794895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:24.809105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62106b-fc12-4a8f-b830-caac729896de None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:24.831998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cf3fe-1962-425b-a1b0-a705787a4594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.832721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cf3fe-1962-425b-a1b0-a705787a4594 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.833276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cf3fe-1962-425b-a1b0-a705787a4594 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.833387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cf3fe-1962-425b-a1b0-a705787a4594 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.834127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cf3fe-1962-425b-a1b0-a705787a4594 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:24.834590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cf3fe-1962-425b-a1b0-a705787a4594 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.836262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cf3fe-1962-425b-a1b0-a705787a4594 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.844305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aa140-f8cf-42d4-b8f1-23002c63ad2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:24.844925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aa140-f8cf-42d4-b8f1-23002c63ad2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:24.845215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aa140-f8cf-42d4-b8f1-23002c63ad2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:24.845486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964aa140-f8cf-42d4-b8f1-23002c63ad2e None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:24.846138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964aa140-f8cf-42d4-b8f1-23002c63ad2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:24.847527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964aa140-f8cf-42d4-b8f1-23002c63ad2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:24.854013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 524/1038] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:46:24.855557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 515/1039] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:24.863613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aa140-f8cf-42d4-b8f1-23002c63ad2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:24.976398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964aa140-f8cf-42d4-b8f1-23002c63ad2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:25.018479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82dd8b-a7ff-402a-921c-48720388eb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:25.018479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82dd8b-a7ff-402a-921c-48720388eb68 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:25.018780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82dd8b-a7ff-402a-921c-48720388eb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:25.018858 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82dd8b-a7ff-402a-921c-48720388eb68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:25.046550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:25.096830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82dd8b-a7ff-402a-921c-48720388eb68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:25.098284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82dd8b-a7ff-402a-921c-48720388eb68 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:25.202495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 525/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:25.208101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cf3fe-1962-425b-a1b0-a705787a4594 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:25.245727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:25.312445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cf3fe-1962-425b-a1b0-a705787a4594 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:25.348375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98dbde-c7d9-4b35-b7ed-096cbeaebb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:25.349016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98dbde-c7d9-4b35-b7ed-096cbeaebb2b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:25.349290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98dbde-c7d9-4b35-b7ed-096cbeaebb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:25.349556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98dbde-c7d9-4b35-b7ed-096cbeaebb2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:25.360610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98dbde-c7d9-4b35-b7ed-096cbeaebb2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:25.362078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98dbde-c7d9-4b35-b7ed-096cbeaebb2b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-dc98dbde-c7d9-4b35-b7ed-096cbeaebb2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:25.376895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:25.377845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:25.379390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:25.379390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:25.379390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:25.379390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:25.379390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 516/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:25.384472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82dd8b-a7ff-402a-921c-48720388eb68 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:25.441580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:25.488249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82dd8b-a7ff-402a-921c-48720388eb68 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:25.522321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71720c6-a542-4810-991a-328f5cd070a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:25.522894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71720c6-a542-4810-991a-328f5cd070a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:25.523179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71720c6-a542-4810-991a-328f5cd070a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:25.523451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71720c6-a542-4810-991a-328f5cd070a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:25.524142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71720c6-a542-4810-991a-328f5cd070a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:25.524443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71720c6-a542-4810-991a-328f5cd070a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:25.526630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71720c6-a542-4810-991a-328f5cd070a0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:25.543301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 526/1042] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:25.544496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98dbde-c7d9-4b35-b7ed-096cbeaebb2b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:25.636847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:25.649869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98dbde-c7d9-4b35-b7ed-096cbeaebb2b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:25.684756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0681e-74d4-4f91-af5b-cc55fcdcf4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:25.685394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0681e-74d4-4f91-af5b-cc55fcdcf4dd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:25.685674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0681e-74d4-4f91-af5b-cc55fcdcf4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:25.685945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0681e-74d4-4f91-af5b-cc55fcdcf4dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:25.692538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0681e-74d4-4f91-af5b-cc55fcdcf4dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:25.693970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0681e-74d4-4f91-af5b-cc55fcdcf4dd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7fa0681e-74d4-4f91-af5b-cc55fcdcf4dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:46:25.705351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:25.706255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:25.707977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:46:25.707977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:46:25.707977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:46:25.707977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:46:25.707977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:46:25.707977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:46:25.707977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:25.707977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:25.707977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 517/1043] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:25.712271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71720c6-a542-4810-991a-328f5cd070a0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:25.817157 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71720c6-a542-4810-991a-328f5cd070a0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:25.830962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:25.852519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a806d98-2ff0-4fa4-8451-52520a0212da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:25.853137 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a806d98-2ff0-4fa4-8451-52520a0212da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:25.853413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a806d98-2ff0-4fa4-8451-52520a0212da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:25.854231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a806d98-2ff0-4fa4-8451-52520a0212da None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:25.854993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a806d98-2ff0-4fa4-8451-52520a0212da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:25.855294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a806d98-2ff0-4fa4-8451-52520a0212da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:25.856764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a806d98-2ff0-4fa4-8451-52520a0212da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:25.871085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 527/1044] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:25.874901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0681e-74d4-4f91-af5b-cc55fcdcf4dd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:25.976959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0681e-74d4-4f91-af5b-cc55fcdcf4dd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:25.977751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39d720-829c-4d55-ba9c-8fb3db1ed58e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:25.978040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39d720-829c-4d55-ba9c-8fb3db1ed58e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:25.978334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39d720-829c-4d55-ba9c-8fb3db1ed58e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:25.980057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 528/1045] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:25.984952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdfefd-9b3a-4ff4-bf02-5ac93779503c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:25.985234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdfefd-9b3a-4ff4-bf02-5ac93779503c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:25.985510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdfefd-9b3a-4ff4-bf02-5ac93779503c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:26.008388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bcfdfefd-9b3a-4ff4-bf02-5ac93779503c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:26.008974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bcfdfefd-9b3a-4ff4-bf02-5ac93779503c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:26.015344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afdf503-7a3e-488b-9f11-569c21098951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:26.015980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdf503-7a3e-488b-9f11-569c21098951 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:26.016257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdf503-7a3e-488b-9f11-569c21098951 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:26.016524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afdf503-7a3e-488b-9f11-569c21098951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:26.017329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afdf503-7a3e-488b-9f11-569c21098951 None admin] RBAC: Adding query filter params (role.id=be2caee164e146c9ab3f7cd86636da45, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=a1dc3ad467d9496cb3e0086a0cd8215e) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:26.017641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afdf503-7a3e-488b-9f11-569c21098951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:26.020805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afdf503-7a3e-488b-9f11-569c21098951 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:26.034476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 518/1046] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:25 2022] GET /identity/v3/role_assignments?user.id=a1dc3ad467d9496cb3e0086a0cd8215e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=be2caee164e146c9ab3f7cd86636da45 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:26.100169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 529/1047] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:26.124020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afdf503-7a3e-488b-9f11-569c21098951 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:26.140572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:26.236816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afdf503-7a3e-488b-9f11-569c21098951 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:26.273886 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa624cfc-5470-4499-992c-b89f8bfa5009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:26.274555 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa624cfc-5470-4499-992c-b89f8bfa5009 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:26.274838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa624cfc-5470-4499-992c-b89f8bfa5009 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:26.275409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa624cfc-5470-4499-992c-b89f8bfa5009 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:26.285876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa624cfc-5470-4499-992c-b89f8bfa5009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-aa624cfc-5470-4499-992c-b89f8bfa5009 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:26.291370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:26.292332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:26.293177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:26.294643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 519/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:46:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:26.298525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdfefd-9b3a-4ff4-bf02-5ac93779503c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:26.401485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdfefd-9b3a-4ff4-bf02-5ac93779503c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:26.403780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:26.437187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730a1f8-05ed-4cdd-baa7-5e3d340b2de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:26.437819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730a1f8-05ed-4cdd-baa7-5e3d340b2de9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:26.438102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730a1f8-05ed-4cdd-baa7-5e3d340b2de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:26.438378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730a1f8-05ed-4cdd-baa7-5e3d340b2de9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:26.439133 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730a1f8-05ed-4cdd-baa7-5e3d340b2de9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:26.439439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730a1f8-05ed-4cdd-baa7-5e3d340b2de9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:26.440819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730a1f8-05ed-4cdd-baa7-5e3d340b2de9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:26.454397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 530/1049] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:46:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:26.455635 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa624cfc-5470-4499-992c-b89f8bfa5009 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:26.529972 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:26.561964 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa624cfc-5470-4499-992c-b89f8bfa5009 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:26.596344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665cc25-4c76-43a2-b828-bc6fe566ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:26.596989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665cc25-4c76-43a2-b828-bc6fe566ab00 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:26.597271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665cc25-4c76-43a2-b828-bc6fe566ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:26.597543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665cc25-4c76-43a2-b828-bc6fe566ab00 None admin] PATH_INFO: `/v3/users/fcbdfe6aea4340d3b0ae669128703add` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:26.611217 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665cc25-4c76-43a2-b828-bc6fe566ab00 None admin] RBAC: Authorizing `identity:get_user(user_id=fcbdfe6aea4340d3b0ae669128703add)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:26.612792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665cc25-4c76-43a2-b828-bc6fe566ab00 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:26.614498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 520/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:26 2022] GET /identity/v3/users/fcbdfe6aea4340d3b0ae669128703add => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:26.620536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730a1f8-05ed-4cdd-baa7-5e3d340b2de9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:26.727780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730a1f8-05ed-4cdd-baa7-5e3d340b2de9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:26.762956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c15fc-79b8-4d25-9c26-faa076de2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:26.763581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c15fc-79b8-4d25-9c26-faa076de2177 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:26.763859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c15fc-79b8-4d25-9c26-faa076de2177 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:26.764130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c15fc-79b8-4d25-9c26-faa076de2177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:26.801268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:26.836845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c15fc-79b8-4d25-9c26-faa076de2177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:26.838869 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c15fc-79b8-4d25-9c26-faa076de2177 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:26.932805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee08f0-5020-4a22-978f-e08171653999 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:26.933149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee08f0-5020-4a22-978f-e08171653999 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:26.933446 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee08f0-5020-4a22-978f-e08171653999 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:26.935025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 521/1051] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:26.939415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81415153-372b-4a49-8928-be6d0ace36ec None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:26.939786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81415153-372b-4a49-8928-be6d0ace36ec None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:26.940129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81415153-372b-4a49-8928-be6d0ace36ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:26.941743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 531/1052] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:26.948706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c15fc-79b8-4d25-9c26-faa076de2177 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:26.964417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-81415153-372b-4a49-8928-be6d0ace36ec None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:26.966247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-81415153-372b-4a49-8928-be6d0ace36ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:27.016876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:27.049475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 522/1053] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:27.061410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c15fc-79b8-4d25-9c26-faa076de2177 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:27.072193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81415153-372b-4a49-8928-be6d0ace36ec None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:27.098790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6430832-08a6-4dee-baca-448e0c2b0a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:27.099437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6430832-08a6-4dee-baca-448e0c2b0a8b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:27.099717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6430832-08a6-4dee-baca-448e0c2b0a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:27.099984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6430832-08a6-4dee-baca-448e0c2b0a8b None admin] PATH_INFO: `/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:27.101143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6430832-08a6-4dee-baca-448e0c2b0a8b None admin] RBAC: Authorizing `identity:get_project(project_id=9fc87c882b2b46fb8ef8d90633d390dc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:27.102594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6430832-08a6-4dee-baca-448e0c2b0a8b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:27.105114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 532/1054] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:46:26 2022] GET /identity/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:27.110840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6430832-08a6-4dee-baca-448e0c2b0a8b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:27.145805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:27.193725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81415153-372b-4a49-8928-be6d0ace36ec None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:27.221338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6430832-08a6-4dee-baca-448e0c2b0a8b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:27.231994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c45fea-300b-4820-aa30-7e10e5aa6ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:27.232995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c45fea-300b-4820-aa30-7e10e5aa6ebc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:27.233278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c45fea-300b-4820-aa30-7e10e5aa6ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:27.233869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c45fea-300b-4820-aa30-7e10e5aa6ebc None admin] PATH_INFO: `/v3/services/image` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:27.234533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c45fea-300b-4820-aa30-7e10e5aa6ebc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:27.235899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c45fea-300b-4820-aa30-7e10e5aa6ebc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d3c45fea-300b-4820-aa30-7e10e5aa6ebc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:27.246376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:27.247529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:27.248394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 523/1055] 10.222.0.20 () {56 vars in 1105 bytes} [Sat Jul 30 04:46:27 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:27.253604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c45fea-300b-4820-aa30-7e10e5aa6ebc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:27.266715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a470e2e-fe02-4f05-841d-28ba49768da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:27.267553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a470e2e-fe02-4f05-841d-28ba49768da7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:27.267782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a470e2e-fe02-4f05-841d-28ba49768da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:27.268057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a470e2e-fe02-4f05-841d-28ba49768da7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:27.268845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a470e2e-fe02-4f05-841d-28ba49768da7 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=9fc87c882b2b46fb8ef8d90633d390dc, user.id=fcbdfe6aea4340d3b0ae669128703add) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:27.269147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a470e2e-fe02-4f05-841d-28ba49768da7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:27.270610 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a470e2e-fe02-4f05-841d-28ba49768da7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:27.272570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:27.291804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 533/1056] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:27 2022] GET /identity/v3/role_assignments?user.id=fcbdfe6aea4340d3b0ae669128703add&scope.project.id=9fc87c882b2b46fb8ef8d90633d390dc&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:27.375270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c45fea-300b-4820-aa30-7e10e5aa6ebc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:27.401164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:27.413817 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a964ac-3c6e-4cf3-ab13-7640edb99222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:27.414473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a964ac-3c6e-4cf3-ab13-7640edb99222 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:27.414736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a964ac-3c6e-4cf3-ab13-7640edb99222 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:27.415005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a964ac-3c6e-4cf3-ab13-7640edb99222 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:27.415687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a964ac-3c6e-4cf3-ab13-7640edb99222 None admin] RBAC: Adding query filter params (name=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:27.416021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a964ac-3c6e-4cf3-ab13-7640edb99222 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:27.417410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a964ac-3c6e-4cf3-ab13-7640edb99222 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:27.422818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 524/1057] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:27.429529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a470e2e-fe02-4f05-841d-28ba49768da7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:27.535843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a470e2e-fe02-4f05-841d-28ba49768da7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:27.570095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db7e43-2c67-4b68-affa-520f67a53cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:27.570751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db7e43-2c67-4b68-affa-520f67a53cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:27.571037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db7e43-2c67-4b68-affa-520f67a53cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:27.571312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db7e43-2c67-4b68-affa-520f67a53cf0 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:27.571991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db7e43-2c67-4b68-affa-520f67a53cf0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:27.572326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db7e43-2c67-4b68-affa-520f67a53cf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:27.573706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47db7e43-2c67-4b68-affa-520f67a53cf0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:27.583338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 534/1058] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:27.584831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a964ac-3c6e-4cf3-ab13-7640edb99222 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:27.687752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a964ac-3c6e-4cf3-ab13-7640edb99222 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:27.722830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acaff6-3509-44cb-bb86-d16e36854a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:27.723456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acaff6-3509-44cb-bb86-d16e36854a25 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:27.723738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acaff6-3509-44cb-bb86-d16e36854a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:27.724007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acaff6-3509-44cb-bb86-d16e36854a25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:27.724729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acaff6-3509-44cb-bb86-d16e36854a25 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:27.726098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acaff6-3509-44cb-bb86-d16e36854a25 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:27.751837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 525/1059] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:46:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:28.070877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df11de-fb8b-494d-80dc-db4ddcbdbcaf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:28.071380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df11de-fb8b-494d-80dc-db4ddcbdbcaf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:28.071458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df11de-fb8b-494d-80dc-db4ddcbdbcaf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:28.073338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 535/1060] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:28.078328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c94416-48af-4058-ba71-a934e3d88fbc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:28.078660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c94416-48af-4058-ba71-a934e3d88fbc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:28.078934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c94416-48af-4058-ba71-a934e3d88fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:28.097411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d1c94416-48af-4058-ba71-a934e3d88fbc None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:28.098624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d1c94416-48af-4058-ba71-a934e3d88fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:28.141692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:28.177757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 526/1061] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:28.200847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df11de-fb8b-494d-80dc-db4ddcbdbcaf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:28.266846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:28.309195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df11de-fb8b-494d-80dc-db4ddcbdbcaf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:28.348058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeaf882-a451-4f71-99d2-f773e85b13bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:28.348707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeaf882-a451-4f71-99d2-f773e85b13bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:28.348987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeaf882-a451-4f71-99d2-f773e85b13bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:28.349257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeaf882-a451-4f71-99d2-f773e85b13bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:28.360536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeaf882-a451-4f71-99d2-f773e85b13bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0aeaf882-a451-4f71-99d2-f773e85b13bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:28.365372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:28.366544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:28.367784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 536/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Jul 30 04:46:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:28.373412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c94416-48af-4058-ba71-a934e3d88fbc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:28.476946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c94416-48af-4058-ba71-a934e3d88fbc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:28.512346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4058e-dd9f-44ae-8808-280c71599c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:28.512966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4058e-dd9f-44ae-8808-280c71599c9e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:28.513244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4058e-dd9f-44ae-8808-280c71599c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:28.513515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4058e-dd9f-44ae-8808-280c71599c9e None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:28.514232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4058e-dd9f-44ae-8808-280c71599c9e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:28.514560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4058e-dd9f-44ae-8808-280c71599c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:28.515925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4058e-dd9f-44ae-8808-280c71599c9e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:28.523929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 527/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Jul 30 04:46:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:28.530781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeaf882-a451-4f71-99d2-f773e85b13bb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:28.632468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeaf882-a451-4f71-99d2-f773e85b13bb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:28.655599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:28.668296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3089210-4a30-446b-9db3-f12486becab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:28.668927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3089210-4a30-446b-9db3-f12486becab3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:28.669204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3089210-4a30-446b-9db3-f12486becab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:28.669471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3089210-4a30-446b-9db3-f12486becab3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:28.679837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3089210-4a30-446b-9db3-f12486becab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:28.681273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3089210-4a30-446b-9db3-f12486becab3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d3089210-4a30-446b-9db3-f12486becab3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:28.696758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:28.697952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:28.703542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:28.703542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:28.703542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:28.703542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:28.703542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 537/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:28.705090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4058e-dd9f-44ae-8808-280c71599c9e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:28.738765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:28.809616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4058e-dd9f-44ae-8808-280c71599c9e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:28.843876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31078d-7e5f-4719-bffe-e084383d8a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:28.844491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31078d-7e5f-4719-bffe-e084383d8a85 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:28.844767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31078d-7e5f-4719-bffe-e084383d8a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:28.845034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31078d-7e5f-4719-bffe-e084383d8a85 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:28.845827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31078d-7e5f-4719-bffe-e084383d8a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:28.846002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31078d-7e5f-4719-bffe-e084383d8a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:28.847389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31078d-7e5f-4719-bffe-e084383d8a85 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:28.860785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 528/1065] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:28.866162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3089210-4a30-446b-9db3-f12486becab3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:28.969622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3089210-4a30-446b-9db3-f12486becab3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:29.003787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca73c54-b692-4084-8a7e-ed42479514d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:29.004428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca73c54-b692-4084-8a7e-ed42479514d1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.004704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca73c54-b692-4084-8a7e-ed42479514d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.004971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca73c54-b692-4084-8a7e-ed42479514d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.016596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca73c54-b692-4084-8a7e-ed42479514d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:29.020205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca73c54-b692-4084-8a7e-ed42479514d1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8ca73c54-b692-4084-8a7e-ed42479514d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:46:29.033442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:29.034419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:29.040274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 538/1066] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Jul 30 04:46:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:29.041097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31078d-7e5f-4719-bffe-e084383d8a85 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:29.135505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:29.144999 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31078d-7e5f-4719-bffe-e084383d8a85 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:29.180900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0ecb9-0914-4ca7-938f-d9d021344c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:29.181550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0ecb9-0914-4ca7-938f-d9d021344c5b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.181830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0ecb9-0914-4ca7-938f-d9d021344c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.182139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0ecb9-0914-4ca7-938f-d9d021344c5b None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.182922 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0ecb9-0914-4ca7-938f-d9d021344c5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:29.183227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0ecb9-0914-4ca7-938f-d9d021344c5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:29.184717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0ecb9-0914-4ca7-938f-d9d021344c5b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:29.206731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 529/1067] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Jul 30 04:46:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:29.207757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca73c54-b692-4084-8a7e-ed42479514d1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:29.244737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:29.310033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52629f9-6fbe-4e1a-8276-469a577f9f38 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.310365 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52629f9-6fbe-4e1a-8276-469a577f9f38 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.310624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52629f9-6fbe-4e1a-8276-469a577f9f38 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.311974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 530/1068] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:29.314806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca73c54-b692-4084-8a7e-ed42479514d1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:29.317042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d417f4-1777-44dc-8bcc-85a9f400e5ee None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.317289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d417f4-1777-44dc-8bcc-85a9f400e5ee None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.317568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d417f4-1777-44dc-8bcc-85a9f400e5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.335934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-79d417f4-1777-44dc-8bcc-85a9f400e5ee None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:29.337134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-79d417f4-1777-44dc-8bcc-85a9f400e5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:29.351910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059c105-6c7e-4930-8b81-4d869377151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:29.352563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059c105-6c7e-4930-8b81-4d869377151e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.352859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059c105-6c7e-4930-8b81-4d869377151e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.353115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059c105-6c7e-4930-8b81-4d869377151e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.364982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059c105-6c7e-4930-8b81-4d869377151e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:29.366421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059c105-6c7e-4930-8b81-4d869377151e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0059c105-6c7e-4930-8b81-4d869377151e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:29.381920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:29.383108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:29.384237 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:29.384237 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:29.384237 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:29.384237 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:29.384237 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 539/1069] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:29.389362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059c105-6c7e-4930-8b81-4d869377151e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:29.418194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 531/1070] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:29.440963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d417f4-1777-44dc-8bcc-85a9f400e5ee None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:29.478994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:29.494559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059c105-6c7e-4930-8b81-4d869377151e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:29.528700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b01d8b-24d2-4275-8b4b-0e40cafc9ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:29.529248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b01d8b-24d2-4275-8b4b-0e40cafc9ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.529526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b01d8b-24d2-4275-8b4b-0e40cafc9ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.529797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b01d8b-24d2-4275-8b4b-0e40cafc9ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.530501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b01d8b-24d2-4275-8b4b-0e40cafc9ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:29.530806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b01d8b-24d2-4275-8b4b-0e40cafc9ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:29.532169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b01d8b-24d2-4275-8b4b-0e40cafc9ca4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:29.533582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:29.545516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 540/1071] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:29.551158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b01d8b-24d2-4275-8b4b-0e40cafc9ca4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:29.563774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d417f4-1777-44dc-8bcc-85a9f400e5ee None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:29.601089 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fce82a-df85-4b03-b20a-efdf176d566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:29.601730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fce82a-df85-4b03-b20a-efdf176d566f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.602009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fce82a-df85-4b03-b20a-efdf176d566f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.602277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fce82a-df85-4b03-b20a-efdf176d566f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.614208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fce82a-df85-4b03-b20a-efdf176d566f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-56fce82a-df85-4b03-b20a-efdf176d566f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:29.619673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:29.620705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:29.622043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 532/1072] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:46:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:29.626582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fce82a-df85-4b03-b20a-efdf176d566f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:29.669626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b01d8b-24d2-4275-8b4b-0e40cafc9ca4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:29.708168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85ecb0-d41e-423a-b925-2c1f19634528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:29.708811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85ecb0-d41e-423a-b925-2c1f19634528 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.709087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85ecb0-d41e-423a-b925-2c1f19634528 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.709359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85ecb0-d41e-423a-b925-2c1f19634528 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.717803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85ecb0-d41e-423a-b925-2c1f19634528 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:29.719791 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85ecb0-d41e-423a-b925-2c1f19634528 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7b85ecb0-d41e-423a-b925-2c1f19634528 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:46:29.732262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:29.733216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:29.734872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:46:29.734872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:46:29.734872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:46:29.734872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:46:29.734872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:46:29.734872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:46:29.734872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:29.734872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:29.734872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 541/1073] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:29.739399 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85ecb0-d41e-423a-b925-2c1f19634528 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:29.740562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fce82a-df85-4b03-b20a-efdf176d566f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:29.748879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:29.782302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91ebbe-3b3d-41bf-9000-e73f556ce8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:29.782876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91ebbe-3b3d-41bf-9000-e73f556ce8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.783156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91ebbe-3b3d-41bf-9000-e73f556ce8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.783430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91ebbe-3b3d-41bf-9000-e73f556ce8a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.784137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91ebbe-3b3d-41bf-9000-e73f556ce8a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:29.784444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91ebbe-3b3d-41bf-9000-e73f556ce8a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:29.785817 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91ebbe-3b3d-41bf-9000-e73f556ce8a0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:29.795810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 533/1074] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:46:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:29.799339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc5990-855f-476c-97d1-64d70b3be955 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.799625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc5990-855f-476c-97d1-64d70b3be955 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.799896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc5990-855f-476c-97d1-64d70b3be955 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.801364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 534/1075] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:46:29.803398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc5990-855f-476c-97d1-64d70b3be955 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:29.806614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:29.848598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85ecb0-d41e-423a-b925-2c1f19634528 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:29.884561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c676f6-69b5-44d6-92d2-ed52c5a2b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:29.885302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c676f6-69b5-44d6-92d2-ed52c5a2b048 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.885586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c676f6-69b5-44d6-92d2-ed52c5a2b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.885868 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c676f6-69b5-44d6-92d2-ed52c5a2b048 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.886652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c676f6-69b5-44d6-92d2-ed52c5a2b048 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:29.886958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c676f6-69b5-44d6-92d2-ed52c5a2b048 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:29.888392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c676f6-69b5-44d6-92d2-ed52c5a2b048 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:29.901113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 542/1076] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:29.902597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f59c7-aa56-4c44-ae65-403eb40e8938 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.902885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f59c7-aa56-4c44-ae65-403eb40e8938 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.903163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f59c7-aa56-4c44-ae65-403eb40e8938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.907766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc5990-855f-476c-97d1-64d70b3be955 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:29.921621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc1f59c7-aa56-4c44-ae65-403eb40e8938 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:29.922848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc1f59c7-aa56-4c44-ae65-403eb40e8938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:29.941999 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34131d79-79ef-4bc0-b661-503286bbde6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:29.942662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34131d79-79ef-4bc0-b661-503286bbde6c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:29.942940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34131d79-79ef-4bc0-b661-503286bbde6c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:29.943208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34131d79-79ef-4bc0-b661-503286bbde6c None admin] PATH_INFO: `/v3/users/fcbdfe6aea4340d3b0ae669128703add` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:29.944471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34131d79-79ef-4bc0-b661-503286bbde6c None admin] RBAC: Authorizing `identity:get_user(user_id=fcbdfe6aea4340d3b0ae669128703add)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:29.945888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34131d79-79ef-4bc0-b661-503286bbde6c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:29.947912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 535/1077] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:29 2022] GET /identity/v3/users/fcbdfe6aea4340d3b0ae669128703add => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:29.948470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34131d79-79ef-4bc0-b661-503286bbde6c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:29.986384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:29.994225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 543/1078] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:29.995067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f59c7-aa56-4c44-ae65-403eb40e8938 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:30.023381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:30.063607 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34131d79-79ef-4bc0-b661-503286bbde6c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:30.097334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f635f5-deec-4d47-95b7-01425ae64283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:30.097988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f635f5-deec-4d47-95b7-01425ae64283 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:30.098260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f635f5-deec-4d47-95b7-01425ae64283 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:30.098559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f635f5-deec-4d47-95b7-01425ae64283 None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/a1dc3ad467d9496cb3e0086a0cd8215e/roles/be2caee164e146c9ab3f7cd86636da45` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:30.098669 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f59c7-aa56-4c44-ae65-403eb40e8938 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:30.100748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f635f5-deec-4d47-95b7-01425ae64283 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=a1dc3ad467d9496cb3e0086a0cd8215e, role_id=be2caee164e146c9ab3f7cd86636da45)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:30.102167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f635f5-deec-4d47-95b7-01425ae64283 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:30.115639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 536/1079] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:46:29 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/a1dc3ad467d9496cb3e0086a0cd8215e/roles/be2caee164e146c9ab3f7cd86636da45 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:46:30.116468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f635f5-deec-4d47-95b7-01425ae64283 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:30.147654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea36ff-f0b7-497f-9c91-848992cd101a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:30.148327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea36ff-f0b7-497f-9c91-848992cd101a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:30.148614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea36ff-f0b7-497f-9c91-848992cd101a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:30.148889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea36ff-f0b7-497f-9c91-848992cd101a None admin] PATH_INFO: `/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:30.151024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea36ff-f0b7-497f-9c91-848992cd101a None admin] RBAC: Authorizing `identity:get_project(project_id=9fc87c882b2b46fb8ef8d90633d390dc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:30.152500 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea36ff-f0b7-497f-9c91-848992cd101a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:30.154308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 544/1080] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:46:29 2022] GET /identity/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:30.160327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea36ff-f0b7-497f-9c91-848992cd101a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:30.229700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:30.244805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f635f5-deec-4d47-95b7-01425ae64283 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:30.281635 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:30.281850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea36ff-f0b7-497f-9c91-848992cd101a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:30.287239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05485e42-e436-4523-a2ab-b2ab94645058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:30.288162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05485e42-e436-4523-a2ab-b2ab94645058 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:30.288447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05485e42-e436-4523-a2ab-b2ab94645058 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:30.288721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05485e42-e436-4523-a2ab-b2ab94645058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:30.317956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962d540-ed17-4dad-98e6-7eed0e959f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:30.318595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962d540-ed17-4dad-98e6-7eed0e959f6f None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:30.318879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962d540-ed17-4dad-98e6-7eed0e959f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:30.319160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962d540-ed17-4dad-98e6-7eed0e959f6f None admin] PATH_INFO: `/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc/users/fcbdfe6aea4340d3b0ae669128703add/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:30.321143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962d540-ed17-4dad-98e6-7eed0e959f6f None admin] RBAC: Authorizing `identity:create_grant(project_id=9fc87c882b2b46fb8ef8d90633d390dc, user_id=fcbdfe6aea4340d3b0ae669128703add, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:30.323620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962d540-ed17-4dad-98e6-7eed0e959f6f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:30.340425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 545/1081] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:46:30 2022] PUT /identity/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc/users/fcbdfe6aea4340d3b0ae669128703add/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:46:30.366859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05485e42-e436-4523-a2ab-b2ab94645058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:30.368346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05485e42-e436-4523-a2ab-b2ab94645058 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:30.484991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 537/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:30.486710 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962d540-ed17-4dad-98e6-7eed0e959f6f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:30.597750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962d540-ed17-4dad-98e6-7eed0e959f6f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:30.622893 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:30.635048 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e9836-306e-4f40-9268-6261a0b810ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:30.635982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e9836-306e-4f40-9268-6261a0b810ae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:30.636456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e9836-306e-4f40-9268-6261a0b810ae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:30.636920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e9836-306e-4f40-9268-6261a0b810ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:30.647594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e9836-306e-4f40-9268-6261a0b810ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:30.649249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e9836-306e-4f40-9268-6261a0b810ae None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9c4e9836-306e-4f40-9268-6261a0b810ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:30.665321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:30.667076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:30.668015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:30.668015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:30.668015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:30.668015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:30.668249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 546/1083] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:30.673830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05485e42-e436-4523-a2ab-b2ab94645058 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:30.705295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:30.780001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05485e42-e436-4523-a2ab-b2ab94645058 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:30.815821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c3d5-12b0-4acd-8f47-c78ebe366cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:30.816439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c3d5-12b0-4acd-8f47-c78ebe366cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:30.816716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c3d5-12b0-4acd-8f47-c78ebe366cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:30.816986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c3d5-12b0-4acd-8f47-c78ebe366cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:30.817676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c3d5-12b0-4acd-8f47-c78ebe366cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:30.817975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c3d5-12b0-4acd-8f47-c78ebe366cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:30.819354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c3d5-12b0-4acd-8f47-c78ebe366cf6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:30.831647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 538/1084] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:32.125932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2dca9-e141-454a-ab84-71d8de290170 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.126507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2dca9-e141-454a-ab84-71d8de290170 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.126507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2dca9-e141-454a-ab84-71d8de290170 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.127872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 547/1085] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:32.133135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2784ffd-27da-4229-a9ba-f7da31336d4b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.133552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2784ffd-27da-4229-a9ba-f7da31336d4b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.133685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2784ffd-27da-4229-a9ba-f7da31336d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.152539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c2784ffd-27da-4229-a9ba-f7da31336d4b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:32.153723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c2784ffd-27da-4229-a9ba-f7da31336d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:32.218158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 539/1086] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:32.240717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2dca9-e141-454a-ab84-71d8de290170 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:32.343116 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2dca9-e141-454a-ab84-71d8de290170 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:32.376374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b91301-e989-4587-9752-c54e77bb3516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:32.377014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b91301-e989-4587-9752-c54e77bb3516 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.377292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b91301-e989-4587-9752-c54e77bb3516 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.377563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b91301-e989-4587-9752-c54e77bb3516 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.381619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3c2d-e72c-41c5-a3a9-98b88d113cdb None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.381907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3c2d-e72c-41c5-a3a9-98b88d113cdb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.382183 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3c2d-e72c-41c5-a3a9-98b88d113cdb None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.383569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 540/1087] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:32.388922 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b91301-e989-4587-9752-c54e77bb3516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:32.389216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbfdd5-82da-435d-a7bf-f5bf1502726d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.389495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbfdd5-82da-435d-a7bf-f5bf1502726d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.389763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbfdd5-82da-435d-a7bf-f5bf1502726d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-96b91301-e989-4587-9752-c54e77bb3516 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:32.393782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:32.394790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:32.395676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 548/1088] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Jul 30 04:46:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:32.400885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b91301-e989-4587-9752-c54e77bb3516 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:32.409684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cefbfdd5-82da-435d-a7bf-f5bf1502726d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:32.410915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cefbfdd5-82da-435d-a7bf-f5bf1502726d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:32.426344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:32.436726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:32.482209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 541/1089] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:32.505170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbfdd5-82da-435d-a7bf-f5bf1502726d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:32.505337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b91301-e989-4587-9752-c54e77bb3516 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:32.538560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123765f8-479a-47bd-834d-91e3248e0dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:32.539179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123765f8-479a-47bd-834d-91e3248e0dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.539452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123765f8-479a-47bd-834d-91e3248e0dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.539722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123765f8-479a-47bd-834d-91e3248e0dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.540428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123765f8-479a-47bd-834d-91e3248e0dc2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:32.541339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123765f8-479a-47bd-834d-91e3248e0dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:32.542717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123765f8-479a-47bd-834d-91e3248e0dc2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:32.550559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 549/1090] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Jul 30 04:46:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:32.556008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123765f8-479a-47bd-834d-91e3248e0dc2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:32.606952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbfdd5-82da-435d-a7bf-f5bf1502726d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:32.641312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329aba1e-9741-42a3-bb9d-4b49febbdb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:32.641946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aba1e-9741-42a3-bb9d-4b49febbdb17 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.642227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aba1e-9741-42a3-bb9d-4b49febbdb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.642536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aba1e-9741-42a3-bb9d-4b49febbdb17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.651791 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:32.653199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aba1e-9741-42a3-bb9d-4b49febbdb17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:32.655723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123765f8-479a-47bd-834d-91e3248e0dc2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-329aba1e-9741-42a3-bb9d-4b49febbdb17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:32.658376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:32.659557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:32.660679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 542/1091] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:46:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:32.665444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329aba1e-9741-42a3-bb9d-4b49febbdb17 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:32.678416 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:32.694989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bc413-079a-4ccd-8e9a-6c5e63e6fc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:32.695633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bc413-079a-4ccd-8e9a-6c5e63e6fc40 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.696069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bc413-079a-4ccd-8e9a-6c5e63e6fc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.696182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bc413-079a-4ccd-8e9a-6c5e63e6fc40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.707678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bc413-079a-4ccd-8e9a-6c5e63e6fc40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:32.709133 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bc413-079a-4ccd-8e9a-6c5e63e6fc40 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-782bc413-079a-4ccd-8e9a-6c5e63e6fc40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:32.724865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:32.725783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:32.726752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:32.726752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:32.726752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:32.726752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:32.726752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 550/1092] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:46:32.732700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bc413-079a-4ccd-8e9a-6c5e63e6fc40 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:32.767189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329aba1e-9741-42a3-bb9d-4b49febbdb17 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:32.803015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228abf22-29de-4661-9655-8241033b8487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:32.803628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228abf22-29de-4661-9655-8241033b8487 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.803903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228abf22-29de-4661-9655-8241033b8487 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.804175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228abf22-29de-4661-9655-8241033b8487 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.804887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228abf22-29de-4661-9655-8241033b8487 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:32.805191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228abf22-29de-4661-9655-8241033b8487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:32.806809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228abf22-29de-4661-9655-8241033b8487 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:32.815187 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 543/1093] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:46:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:32.820806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228abf22-29de-4661-9655-8241033b8487 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:32.832169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bc413-079a-4ccd-8e9a-6c5e63e6fc40 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:32.866778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6f4c6-cd16-4311-b3c2-66c5b79c2129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:32.867321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6f4c6-cd16-4311-b3c2-66c5b79c2129 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.867599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6f4c6-cd16-4311-b3c2-66c5b79c2129 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.867868 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6f4c6-cd16-4311-b3c2-66c5b79c2129 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.868547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6f4c6-cd16-4311-b3c2-66c5b79c2129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:32.868958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6f4c6-cd16-4311-b3c2-66c5b79c2129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:32.870381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6f4c6-cd16-4311-b3c2-66c5b79c2129 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:32.886762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 551/1094] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:32.887771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6f4c6-cd16-4311-b3c2-66c5b79c2129 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:32.889841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:32.899561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:32.921253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228abf22-29de-4661-9655-8241033b8487 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:32.954101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb18713-1d66-4deb-b3cd-552d4395aa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:32.954870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb18713-1d66-4deb-b3cd-552d4395aa02 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:32.955223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb18713-1d66-4deb-b3cd-552d4395aa02 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:32.955574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb18713-1d66-4deb-b3cd-552d4395aa02 None admin] PATH_INFO: `/v3/users/fcbdfe6aea4340d3b0ae669128703add` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:32.956766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb18713-1d66-4deb-b3cd-552d4395aa02 None admin] RBAC: Authorizing `identity:get_user(user_id=fcbdfe6aea4340d3b0ae669128703add)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:32.958203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb18713-1d66-4deb-b3cd-552d4395aa02 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:32.959860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 544/1095] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:32 2022] GET /identity/v3/users/fcbdfe6aea4340d3b0ae669128703add => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:32.966464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb18713-1d66-4deb-b3cd-552d4395aa02 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:32.989388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6f4c6-cd16-4311-b3c2-66c5b79c2129 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:33.024032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8685-7d2b-4195-a6ff-4f7369f606f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:33.025555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8685-7d2b-4195-a6ff-4f7369f606f0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:33.026108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8685-7d2b-4195-a6ff-4f7369f606f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:33.026108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8685-7d2b-4195-a6ff-4f7369f606f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:33.033245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd8685-7d2b-4195-a6ff-4f7369f606f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:33.034748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd8685-7d2b-4195-a6ff-4f7369f606f0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-33dd8685-7d2b-4195-a6ff-4f7369f606f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:46:33.046984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:33.047990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:33.048950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 552/1096] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Jul 30 04:46:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:33.053792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8685-7d2b-4195-a6ff-4f7369f606f0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:33.075774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb18713-1d66-4deb-b3cd-552d4395aa02 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:33.109078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4573c5-8add-44f6-a799-fa7f9f243b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:33.109695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4573c5-8add-44f6-a799-fa7f9f243b1a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:33.109974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4573c5-8add-44f6-a799-fa7f9f243b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:33.110246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4573c5-8add-44f6-a799-fa7f9f243b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:33.124144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:33.124502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:33.155906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8685-7d2b-4195-a6ff-4f7369f606f0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:33.192235 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7595-13e2-45d3-9112-540e0d7af6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:33.193174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7595-13e2-45d3-9112-540e0d7af6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:33.193630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7595-13e2-45d3-9112-540e0d7af6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:33.194094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7595-13e2-45d3-9112-540e0d7af6d4 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:33.194910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b7595-13e2-45d3-9112-540e0d7af6d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:33.195218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b7595-13e2-45d3-9112-540e0d7af6d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:33.196639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b7595-13e2-45d3-9112-540e0d7af6d4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:33.197189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4573c5-8add-44f6-a799-fa7f9f243b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:33.199308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4573c5-8add-44f6-a799-fa7f9f243b1a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:33.219263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 553/1097] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Jul 30 04:46:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:33.220251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7595-13e2-45d3-9112-540e0d7af6d4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:33.305048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 545/1098] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 30 04:46:33.311761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4573c5-8add-44f6-a799-fa7f9f243b1a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:33.328207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7595-13e2-45d3-9112-540e0d7af6d4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:33.357930 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:33.360398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:33.368071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d319782-cca0-4f07-89e1-5a7e54ee509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:33.368750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d319782-cca0-4f07-89e1-5a7e54ee509d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:33.369079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d319782-cca0-4f07-89e1-5a7e54ee509d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:33.369441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d319782-cca0-4f07-89e1-5a7e54ee509d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:33.428470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4573c5-8add-44f6-a799-fa7f9f243b1a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:33.443733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d319782-cca0-4f07-89e1-5a7e54ee509d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:33.445194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d319782-cca0-4f07-89e1-5a7e54ee509d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:33.463246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e7255-9023-41d1-8592-758764dd43d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:33.463885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e7255-9023-41d1-8592-758764dd43d1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:33.464169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e7255-9023-41d1-8592-758764dd43d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:33.464432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e7255-9023-41d1-8592-758764dd43d1 None admin] PATH_INFO: `/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:33.465588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e7255-9023-41d1-8592-758764dd43d1 None admin] RBAC: Authorizing `identity:get_project(project_id=9fc87c882b2b46fb8ef8d90633d390dc)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:33.467036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e7255-9023-41d1-8592-758764dd43d1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:33.468693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 546/1099] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:46:33 2022] GET /identity/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:33.474610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e7255-9023-41d1-8592-758764dd43d1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:33.550809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 554/1100] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:33.557042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d319782-cca0-4f07-89e1-5a7e54ee509d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:33.572433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e7255-9023-41d1-8592-758764dd43d1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:33.581942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:33.583143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:33.608701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e992c-e4a9-4511-bad0-800aaa567f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:33.609315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e992c-e4a9-4511-bad0-800aaa567f3b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:33.609595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e992c-e4a9-4511-bad0-800aaa567f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:33.609865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e992c-e4a9-4511-bad0-800aaa567f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:33.610670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e992c-e4a9-4511-bad0-800aaa567f3b None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=9fc87c882b2b46fb8ef8d90633d390dc, user.id=fcbdfe6aea4340d3b0ae669128703add) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:33.610976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e992c-e4a9-4511-bad0-800aaa567f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:33.612400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e992c-e4a9-4511-bad0-800aaa567f3b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:33.618225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 547/1101] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:33 2022] GET /identity/v3/role_assignments?user.id=fcbdfe6aea4340d3b0ae669128703add&scope.project.id=9fc87c882b2b46fb8ef8d90633d390dc&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:33.659575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d319782-cca0-4f07-89e1-5a7e54ee509d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:33.696009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f12833-6072-4eb4-ae8a-773c2cffd3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:33.696560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f12833-6072-4eb4-ae8a-773c2cffd3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:33.697054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f12833-6072-4eb4-ae8a-773c2cffd3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:33.697445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f12833-6072-4eb4-ae8a-773c2cffd3d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:33.709098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f12833-6072-4eb4-ae8a-773c2cffd3d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:33.710629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f12833-6072-4eb4-ae8a-773c2cffd3d6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a8f12833-6072-4eb4-ae8a-773c2cffd3d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:46:33.726837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:33.728618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:33.729520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:46:33.729520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:46:33.729520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:46:33.729520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:33.729520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 555/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:33.734860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e992c-e4a9-4511-bad0-800aaa567f3b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:33.840362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e992c-e4a9-4511-bad0-800aaa567f3b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:33.873696 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db18b-4f01-47f6-b530-b1d64bb6b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:33.874315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db18b-4f01-47f6-b530-b1d64bb6b26e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:33.874641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db18b-4f01-47f6-b530-b1d64bb6b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:33.874914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db18b-4f01-47f6-b530-b1d64bb6b26e None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:33.875604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db18b-4f01-47f6-b530-b1d64bb6b26e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:33.875907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db18b-4f01-47f6-b530-b1d64bb6b26e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:33.877265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db18b-4f01-47f6-b530-b1d64bb6b26e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:33.889946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 548/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:33.895468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f12833-6072-4eb4-ae8a-773c2cffd3d6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:33.979197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:33.999652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f12833-6072-4eb4-ae8a-773c2cffd3d6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:34.034966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548748a1-cafd-4da5-b410-c7bb0c00f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:34.035603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548748a1-cafd-4da5-b410-c7bb0c00f042 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:34.035880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548748a1-cafd-4da5-b410-c7bb0c00f042 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:34.036151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548748a1-cafd-4da5-b410-c7bb0c00f042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:34.042901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548748a1-cafd-4da5-b410-c7bb0c00f042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:34.044360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548748a1-cafd-4da5-b410-c7bb0c00f042 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-548748a1-cafd-4da5-b410-c7bb0c00f042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:46:34.055442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:34.056515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:34.057793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:46:34.057793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:46:34.057793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:46:34.057793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:46:34.057793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:46:34.057793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:46:34.057793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:46:34.057793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:34.057793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 556/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:34.062319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db18b-4f01-47f6-b530-b1d64bb6b26e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:34.093698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:34.163224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db18b-4f01-47f6-b530-b1d64bb6b26e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:34.196268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964485c-6c98-42cf-bc4d-d2500c829645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:34.197442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964485c-6c98-42cf-bc4d-d2500c829645 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:34.198040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964485c-6c98-42cf-bc4d-d2500c829645 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:34.198343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964485c-6c98-42cf-bc4d-d2500c829645 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:34.199164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964485c-6c98-42cf-bc4d-d2500c829645 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:34.199537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964485c-6c98-42cf-bc4d-d2500c829645 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:34.200942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964485c-6c98-42cf-bc4d-d2500c829645 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:34.212898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 549/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:46:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:34.218993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548748a1-cafd-4da5-b410-c7bb0c00f042 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:34.317297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548748a1-cafd-4da5-b410-c7bb0c00f042 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:34.350023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd5148-ab01-475c-a584-f61302572386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:34.350735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd5148-ab01-475c-a584-f61302572386 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:34.351015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd5148-ab01-475c-a584-f61302572386 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:34.351284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd5148-ab01-475c-a584-f61302572386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:34.352061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd5148-ab01-475c-a584-f61302572386 None admin] RBAC: Adding query filter params (role.id=be2caee164e146c9ab3f7cd86636da45, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=a1dc3ad467d9496cb3e0086a0cd8215e) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:34.352366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd5148-ab01-475c-a584-f61302572386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:34.354393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd5148-ab01-475c-a584-f61302572386 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:34.370473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 557/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:34 2022] GET /identity/v3/role_assignments?user.id=a1dc3ad467d9496cb3e0086a0cd8215e&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=be2caee164e146c9ab3f7cd86636da45 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:35.649515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26330893-35d4-4141-a427-740f763c82ca None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:35.650544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26330893-35d4-4141-a427-740f763c82ca None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:35.650544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26330893-35d4-4141-a427-740f763c82ca None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:35.651748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 550/1107] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:35.656825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614be11-0416-4318-85a5-e24e5d3853af None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:35.657239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614be11-0416-4318-85a5-e24e5d3853af None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:35.657357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614be11-0416-4318-85a5-e24e5d3853af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:35.664585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:35.676184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0614be11-0416-4318-85a5-e24e5d3853af None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:35.677363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0614be11-0416-4318-85a5-e24e5d3853af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:35.744247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 558/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:35.768129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26330893-35d4-4141-a427-740f763c82ca None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:35.864147 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:35.874665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26330893-35d4-4141-a427-740f763c82ca None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:35.907227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9e94f-0972-4ad6-8eaa-0298ff0eff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:35.907786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9e94f-0972-4ad6-8eaa-0298ff0eff81 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:35.908058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9e94f-0972-4ad6-8eaa-0298ff0eff81 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:35.908322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9e94f-0972-4ad6-8eaa-0298ff0eff81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:35.909445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9e94f-0972-4ad6-8eaa-0298ff0eff81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:35.910889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9e94f-0972-4ad6-8eaa-0298ff0eff81 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:35.912449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 551/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:35.918426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614be11-0416-4318-85a5-e24e5d3853af None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:36.015341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614be11-0416-4318-85a5-e24e5d3853af None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:36.049081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba948e-4f45-4d98-9093-924f3cdfc9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:36.049689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba948e-4f45-4d98-9093-924f3cdfc9de None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:36.049965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba948e-4f45-4d98-9093-924f3cdfc9de None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:36.050232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba948e-4f45-4d98-9093-924f3cdfc9de None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:36.050945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba948e-4f45-4d98-9093-924f3cdfc9de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:36.052378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba948e-4f45-4d98-9093-924f3cdfc9de None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:36.060531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:36.074510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 559/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:46:35 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:38.101303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02da9f4-7f14-4cb2-86a3-c33a6f4e0c71 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:38.101813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02da9f4-7f14-4cb2-86a3-c33a6f4e0c71 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:38.101905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02da9f4-7f14-4cb2-86a3-c33a6f4e0c71 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:38.104031 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 552/1111] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:38.109143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4585f00-9e49-4989-bc51-683a41bb52ff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:38.109521 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4585f00-9e49-4989-bc51-683a41bb52ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:38.109697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4585f00-9e49-4989-bc51-683a41bb52ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:38.127638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b4585f00-9e49-4989-bc51-683a41bb52ff None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:38.128896 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4585f00-9e49-4989-bc51-683a41bb52ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:38.193466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 560/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:38.216626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02da9f4-7f14-4cb2-86a3-c33a6f4e0c71 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:38.316632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02da9f4-7f14-4cb2-86a3-c33a6f4e0c71 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:38.349559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f125cb-cf32-4fc1-950d-69ee0c2b5722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:38.350187 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f125cb-cf32-4fc1-950d-69ee0c2b5722 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:38.350486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f125cb-cf32-4fc1-950d-69ee0c2b5722 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:38.350775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f125cb-cf32-4fc1-950d-69ee0c2b5722 None admin] PATH_INFO: `/v3/roles/9ec4568f149c40c38cd07fb317a40280` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:38.352240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f125cb-cf32-4fc1-950d-69ee0c2b5722 None admin] RBAC: Authorizing `identity:get_role(role_id=9ec4568f149c40c38cd07fb317a40280)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:38.354517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f125cb-cf32-4fc1-950d-69ee0c2b5722 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:38.355848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 553/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:38 2022] GET /identity/v3/roles/9ec4568f149c40c38cd07fb317a40280 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:38.361601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4585f00-9e49-4989-bc51-683a41bb52ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:38.457779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4585f00-9e49-4989-bc51-683a41bb52ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:38.477526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:38.492581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759ebdd-af1d-40bb-8eca-c1ca79417a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:38.493803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759ebdd-af1d-40bb-8eca-c1ca79417a10 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:38.494082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759ebdd-af1d-40bb-8eca-c1ca79417a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:38.494463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759ebdd-af1d-40bb-8eca-c1ca79417a10 None admin] PATH_INFO: `/v3/users/e8d2b8f023e14c1b9965db07c0778ce1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:38.507263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759ebdd-af1d-40bb-8eca-c1ca79417a10 None admin] RBAC: Authorizing `identity:get_user(user_id=e8d2b8f023e14c1b9965db07c0778ce1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:38.508694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759ebdd-af1d-40bb-8eca-c1ca79417a10 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:38.510450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 561/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:38 2022] GET /identity/v3/users/e8d2b8f023e14c1b9965db07c0778ce1 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:38.516087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f125cb-cf32-4fc1-950d-69ee0c2b5722 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:38.545233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:38.614814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f125cb-cf32-4fc1-950d-69ee0c2b5722 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:38.647261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67220d6-4311-4d97-a798-2210633ff279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:38.647857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67220d6-4311-4d97-a798-2210633ff279 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:38.648137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67220d6-4311-4d97-a798-2210633ff279 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:38.648404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67220d6-4311-4d97-a798-2210633ff279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:38.715524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67220d6-4311-4d97-a798-2210633ff279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:38.716957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67220d6-4311-4d97-a798-2210633ff279 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:38.743979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:38.815255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 554/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:38.821172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759ebdd-af1d-40bb-8eca-c1ca79417a10 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:38.919593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759ebdd-af1d-40bb-8eca-c1ca79417a10 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:38.952161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7a2c0-5b6e-4da2-80de-fc9333eb3f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:38.952786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7a2c0-5b6e-4da2-80de-fc9333eb3f47 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:38.953065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7a2c0-5b6e-4da2-80de-fc9333eb3f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:38.953338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7a2c0-5b6e-4da2-80de-fc9333eb3f47 None admin] PATH_INFO: `/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:38.954571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7a2c0-5b6e-4da2-80de-fc9333eb3f47 None admin] RBAC: Authorizing `identity:get_project(project_id=9fc87c882b2b46fb8ef8d90633d390dc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:38.955986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7a2c0-5b6e-4da2-80de-fc9333eb3f47 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:38.957614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 562/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:46:38 2022] GET /identity/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:38.964371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67220d6-4311-4d97-a798-2210633ff279 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:39.069498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67220d6-4311-4d97-a798-2210633ff279 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:39.099959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:39.108752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73060917-0d11-4ff9-89a7-399c7b575266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:39.109368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73060917-0d11-4ff9-89a7-399c7b575266 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:39.109649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73060917-0d11-4ff9-89a7-399c7b575266 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:39.109924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73060917-0d11-4ff9-89a7-399c7b575266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:39.110739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73060917-0d11-4ff9-89a7-399c7b575266 None admin] RBAC: Adding query filter params (role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=9fc87c882b2b46fb8ef8d90633d390dc, user.id=e8d2b8f023e14c1b9965db07c0778ce1) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:39.111050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73060917-0d11-4ff9-89a7-399c7b575266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:39.112478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73060917-0d11-4ff9-89a7-399c7b575266 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:39.117786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 555/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:38 2022] GET /identity/v3/role_assignments?user.id=e8d2b8f023e14c1b9965db07c0778ce1&scope.project.id=9fc87c882b2b46fb8ef8d90633d390dc&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:41.058689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8b941-05cc-4efb-8d8f-cefed0cfea59 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:41.059148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8b941-05cc-4efb-8d8f-cefed0cfea59 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:41.059240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8b941-05cc-4efb-8d8f-cefed0cfea59 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:41.060582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 563/1118] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:41.065569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4896a-24cf-4f23-8b1c-46985eeea211 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:41.065955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4896a-24cf-4f23-8b1c-46985eeea211 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:41.066123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4896a-24cf-4f23-8b1c-46985eeea211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:41.083981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-73c4896a-24cf-4f23-8b1c-46985eeea211 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:41.085165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-73c4896a-24cf-4f23-8b1c-46985eeea211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:41.150487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 556/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:41.172468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8b941-05cc-4efb-8d8f-cefed0cfea59 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:41.219007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:41.272043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8b941-05cc-4efb-8d8f-cefed0cfea59 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:41.305632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613a4a6-a46e-450a-a00c-5c72b55e6aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:41.306296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613a4a6-a46e-450a-a00c-5c72b55e6aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:41.306603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613a4a6-a46e-450a-a00c-5c72b55e6aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:41.306876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613a4a6-a46e-450a-a00c-5c72b55e6aa3 None admin] PATH_INFO: `/v3/roles/9ec4568f149c40c38cd07fb317a40280` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:41.308297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613a4a6-a46e-450a-a00c-5c72b55e6aa3 None admin] RBAC: Authorizing `identity:get_role(role_id=9ec4568f149c40c38cd07fb317a40280)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:41.309649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613a4a6-a46e-450a-a00c-5c72b55e6aa3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:41.310980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 564/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:41 2022] GET /identity/v3/roles/9ec4568f149c40c38cd07fb317a40280 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:41.316790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4896a-24cf-4f23-8b1c-46985eeea211 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:41.414094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4896a-24cf-4f23-8b1c-46985eeea211 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:41.435467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:41.447497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60c618-dfd4-46af-af74-ab757ea6d688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:41.448130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60c618-dfd4-46af-af74-ab757ea6d688 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:41.448408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60c618-dfd4-46af-af74-ab757ea6d688 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:41.448682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60c618-dfd4-46af-af74-ab757ea6d688 None admin] PATH_INFO: `/v3/users/e8d2b8f023e14c1b9965db07c0778ce1` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:41.449844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60c618-dfd4-46af-af74-ab757ea6d688 None admin] RBAC: Authorizing `identity:get_user(user_id=e8d2b8f023e14c1b9965db07c0778ce1)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:41.451314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60c618-dfd4-46af-af74-ab757ea6d688 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:41.453078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 557/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:41 2022] GET /identity/v3/users/e8d2b8f023e14c1b9965db07c0778ce1 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:41.458790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613a4a6-a46e-450a-a00c-5c72b55e6aa3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:41.559247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613a4a6-a46e-450a-a00c-5c72b55e6aa3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:41.573798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:41.593876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17524b77-6ee8-4415-bab1-f359d348df9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:41.594540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17524b77-6ee8-4415-bab1-f359d348df9d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:41.594821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17524b77-6ee8-4415-bab1-f359d348df9d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:41.595095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17524b77-6ee8-4415-bab1-f359d348df9d None admin] PATH_INFO: `/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:41.596240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17524b77-6ee8-4415-bab1-f359d348df9d None admin] RBAC: Authorizing `identity:get_project(project_id=9fc87c882b2b46fb8ef8d90633d390dc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:41.597661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17524b77-6ee8-4415-bab1-f359d348df9d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:41.599326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 565/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:46:41 2022] GET /identity/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:41.605826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60c618-dfd4-46af-af74-ab757ea6d688 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:41.704985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60c618-dfd4-46af-af74-ab757ea6d688 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:41.738614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2efe30-23ab-450e-9178-4c66f1f6a02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:41.739261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2efe30-23ab-450e-9178-4c66f1f6a02b None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:41.739537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2efe30-23ab-450e-9178-4c66f1f6a02b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:41.739808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2efe30-23ab-450e-9178-4c66f1f6a02b None admin] PATH_INFO: `/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc/users/e8d2b8f023e14c1b9965db07c0778ce1/roles/9ec4568f149c40c38cd07fb317a40280` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:41.741960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2efe30-23ab-450e-9178-4c66f1f6a02b None admin] RBAC: Authorizing `identity:create_grant(project_id=9fc87c882b2b46fb8ef8d90633d390dc, user_id=e8d2b8f023e14c1b9965db07c0778ce1, role_id=9ec4568f149c40c38cd07fb317a40280)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:41.744498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2efe30-23ab-450e-9178-4c66f1f6a02b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:41.756851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 558/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:46:41 2022] PUT /identity/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc/users/e8d2b8f023e14c1b9965db07c0778ce1/roles/9ec4568f149c40c38cd07fb317a40280 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:46:43.718974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d6bf-f1a9-4220-8ddb-3c1802dc82e2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:43.719551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d6bf-f1a9-4220-8ddb-3c1802dc82e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:43.719551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69d6bf-f1a9-4220-8ddb-3c1802dc82e2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:43.720917 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 566/1124] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:43.725943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397757b-9f3d-40b0-b3cf-5855c62c81f0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:43.726383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397757b-9f3d-40b0-b3cf-5855c62c81f0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:43.726579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397757b-9f3d-40b0-b3cf-5855c62c81f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:43.744039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1397757b-9f3d-40b0-b3cf-5855c62c81f0 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:43.745221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1397757b-9f3d-40b0-b3cf-5855c62c81f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:43.790637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:43.811740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 559/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 30 04:46:43.834999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69d6bf-f1a9-4220-8ddb-3c1802dc82e2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:43.937292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69d6bf-f1a9-4220-8ddb-3c1802dc82e2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:43.971232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859592a4-d93b-4d20-ac25-4cb5cb0e210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:43.971811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859592a4-d93b-4d20-ac25-4cb5cb0e210a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:43.972090 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859592a4-d93b-4d20-ac25-4cb5cb0e210a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:43.972364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859592a4-d93b-4d20-ac25-4cb5cb0e210a None admin] PATH_INFO: `/v3/roles/9ec4568f149c40c38cd07fb317a40280` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:43.973814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859592a4-d93b-4d20-ac25-4cb5cb0e210a None admin] RBAC: Authorizing `identity:get_role(role_id=9ec4568f149c40c38cd07fb317a40280)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:43.975201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859592a4-d93b-4d20-ac25-4cb5cb0e210a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:43.976534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 567/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:43 2022] GET /identity/v3/roles/9ec4568f149c40c38cd07fb317a40280 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:43.982640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397757b-9f3d-40b0-b3cf-5855c62c81f0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:44.082715 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397757b-9f3d-40b0-b3cf-5855c62c81f0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:44.115841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51242a50-4113-4f14-88d1-6f1bc0e088e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:44.116473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51242a50-4113-4f14-88d1-6f1bc0e088e3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:44.116755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51242a50-4113-4f14-88d1-6f1bc0e088e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:44.117022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51242a50-4113-4f14-88d1-6f1bc0e088e3 None admin] PATH_INFO: `/v3/users/e8d2b8f023e14c1b9965db07c0778ce1` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:44.119203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51242a50-4113-4f14-88d1-6f1bc0e088e3 None admin] RBAC: Authorizing `identity:get_user(user_id=e8d2b8f023e14c1b9965db07c0778ce1)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:44.120626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51242a50-4113-4f14-88d1-6f1bc0e088e3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:44.122223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 560/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:43 2022] GET /identity/v3/users/e8d2b8f023e14c1b9965db07c0778ce1 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:44.128007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859592a4-d93b-4d20-ac25-4cb5cb0e210a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:44.177045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:44.226504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859592a4-d93b-4d20-ac25-4cb5cb0e210a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:44.260023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d22fc5-b2c8-4537-98ce-93761aec8403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:44.260632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d22fc5-b2c8-4537-98ce-93761aec8403 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:44.260906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d22fc5-b2c8-4537-98ce-93761aec8403 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:44.261175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d22fc5-b2c8-4537-98ce-93761aec8403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:44.327076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d22fc5-b2c8-4537-98ce-93761aec8403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:44.328519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d22fc5-b2c8-4537-98ce-93761aec8403 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:44.374761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:44.424602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 568/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:44.431256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51242a50-4113-4f14-88d1-6f1bc0e088e3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:44.480402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:44.530300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51242a50-4113-4f14-88d1-6f1bc0e088e3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:44.564033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d26aa-20fd-4507-8e77-a91fe48adf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:44.564663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d26aa-20fd-4507-8e77-a91fe48adf1d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:44.564943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d26aa-20fd-4507-8e77-a91fe48adf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:44.565222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d26aa-20fd-4507-8e77-a91fe48adf1d None admin] PATH_INFO: `/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:44.566373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d26aa-20fd-4507-8e77-a91fe48adf1d None admin] RBAC: Authorizing `identity:get_project(project_id=9fc87c882b2b46fb8ef8d90633d390dc)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:44.567827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d26aa-20fd-4507-8e77-a91fe48adf1d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:44.569484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 561/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:46:44 2022] GET /identity/v3/projects/9fc87c882b2b46fb8ef8d90633d390dc => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:44.576029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d22fc5-b2c8-4537-98ce-93761aec8403 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:44.673275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d22fc5-b2c8-4537-98ce-93761aec8403 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:44.707364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490847c8-cc80-4904-b42e-6ed7df9215df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:44.707968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490847c8-cc80-4904-b42e-6ed7df9215df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:44.708251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490847c8-cc80-4904-b42e-6ed7df9215df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:44.708527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490847c8-cc80-4904-b42e-6ed7df9215df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:44.709306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490847c8-cc80-4904-b42e-6ed7df9215df None admin] RBAC: Adding query filter params (role.id=9ec4568f149c40c38cd07fb317a40280, scope.project.id=9fc87c882b2b46fb8ef8d90633d390dc, user.id=e8d2b8f023e14c1b9965db07c0778ce1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:44.709619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490847c8-cc80-4904-b42e-6ed7df9215df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:44.711096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490847c8-cc80-4904-b42e-6ed7df9215df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:44.717027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 569/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:44 2022] GET /identity/v3/role_assignments?user.id=e8d2b8f023e14c1b9965db07c0778ce1&scope.project.id=9fc87c882b2b46fb8ef8d90633d390dc&role.id=9ec4568f149c40c38cd07fb317a40280 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:46.708263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45a85b-3f2d-4db3-b1d8-5e478f858034 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:46.708791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45a85b-3f2d-4db3-b1d8-5e478f858034 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:46.708791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45a85b-3f2d-4db3-b1d8-5e478f858034 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:46.710133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 562/1131] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:46.716316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5282b-91bb-4cbf-9299-812cecb403ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:46.716738 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5282b-91bb-4cbf-9299-812cecb403ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:46.716866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5282b-91bb-4cbf-9299-812cecb403ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:46.736247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-32f5282b-91bb-4cbf-9299-812cecb403ef None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:46.737445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-32f5282b-91bb-4cbf-9299-812cecb403ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:46.746840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:46.803870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 570/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:46.826796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45a85b-3f2d-4db3-b1d8-5e478f858034 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:46.926186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45a85b-3f2d-4db3-b1d8-5e478f858034 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:46.942067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:46.960329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c449689-844a-4b5f-9288-e3817fab5936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:46.960892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c449689-844a-4b5f-9288-e3817fab5936 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:46.962207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c449689-844a-4b5f-9288-e3817fab5936 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:46.962512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c449689-844a-4b5f-9288-e3817fab5936 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:46.963661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c449689-844a-4b5f-9288-e3817fab5936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:46.965079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c449689-844a-4b5f-9288-e3817fab5936 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:46.966659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 563/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:46.972420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5282b-91bb-4cbf-9299-812cecb403ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:47.079344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5282b-91bb-4cbf-9299-812cecb403ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:47.114505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59551cfb-c24d-45e3-85d6-8ba6b4e2dd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:47.115123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59551cfb-c24d-45e3-85d6-8ba6b4e2dd4b None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:47.115401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59551cfb-c24d-45e3-85d6-8ba6b4e2dd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:47.115673 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59551cfb-c24d-45e3-85d6-8ba6b4e2dd4b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:47.116359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59551cfb-c24d-45e3-85d6-8ba6b4e2dd4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:47.118075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59551cfb-c24d-45e3-85d6-8ba6b4e2dd4b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:47.132676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:47.144941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 571/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:46:46 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:46:49.159182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f42ed1-9371-4b7e-bc2e-d5b6efb339bb None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:49.159698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f42ed1-9371-4b7e-bc2e-d5b6efb339bb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:49.159810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f42ed1-9371-4b7e-bc2e-d5b6efb339bb None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:49.161071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 564/1135] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:49.166332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df831d-0564-45a7-9f87-ee249fe2a1ec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:49.166803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df831d-0564-45a7-9f87-ee249fe2a1ec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:49.166899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df831d-0564-45a7-9f87-ee249fe2a1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:49.184735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-60df831d-0564-45a7-9f87-ee249fe2a1ec None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:49.185909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60df831d-0564-45a7-9f87-ee249fe2a1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:49.251961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 572/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:49.277077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f42ed1-9371-4b7e-bc2e-d5b6efb339bb None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:49.380411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f42ed1-9371-4b7e-bc2e-d5b6efb339bb None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:49.417940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f0c91-7deb-42d4-8b14-faa918a3ee6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:49.418597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f0c91-7deb-42d4-8b14-faa918a3ee6f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:49.418875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f0c91-7deb-42d4-8b14-faa918a3ee6f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:49.419146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f0c91-7deb-42d4-8b14-faa918a3ee6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:49.420288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f0c91-7deb-42d4-8b14-faa918a3ee6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:49.421701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f0c91-7deb-42d4-8b14-faa918a3ee6f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:49.423294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 565/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:46:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:49.429061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df831d-0564-45a7-9f87-ee249fe2a1ec None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:49.526119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df831d-0564-45a7-9f87-ee249fe2a1ec None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:49.550582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:49.559789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57829e-81e3-415c-acae-e051a5469ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:49.560393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57829e-81e3-415c-acae-e051a5469ea9 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:49.560669 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57829e-81e3-415c-acae-e051a5469ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:49.560936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57829e-81e3-415c-acae-e051a5469ea9 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:49.561617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57829e-81e3-415c-acae-e051a5469ea9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:49.563074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57829e-81e3-415c-acae-e051a5469ea9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:49.583623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 573/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:46:49 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:46:51.625420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f016a1b-5958-4e34-91b5-35f00a3dbc61 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:51.625953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f016a1b-5958-4e34-91b5-35f00a3dbc61 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:51.625953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f016a1b-5958-4e34-91b5-35f00a3dbc61 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:51.627343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 566/1139] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:51.632568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511402a-7606-40b9-a287-27d6c74b7da5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:51.633182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511402a-7606-40b9-a287-27d6c74b7da5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:51.633182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511402a-7606-40b9-a287-27d6c74b7da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:51.651761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2511402a-7606-40b9-a287-27d6c74b7da5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:51.652948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2511402a-7606-40b9-a287-27d6c74b7da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:51.720492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 574/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:51.744059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f016a1b-5958-4e34-91b5-35f00a3dbc61 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:51.798250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:51.846223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f016a1b-5958-4e34-91b5-35f00a3dbc61 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:51.884297 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df759e8a-d7c4-44bb-a3f1-19aa7635f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:51.885321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df759e8a-d7c4-44bb-a3f1-19aa7635f46d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:51.885830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df759e8a-d7c4-44bb-a3f1-19aa7635f46d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:51.886322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df759e8a-d7c4-44bb-a3f1-19aa7635f46d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:51.897202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df759e8a-d7c4-44bb-a3f1-19aa7635f46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-df759e8a-d7c4-44bb-a3f1-19aa7635f46d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:51.903301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:51.904504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:51.905632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 567/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:46:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:51.912562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511402a-7606-40b9-a287-27d6c74b7da5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:52.024229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511402a-7606-40b9-a287-27d6c74b7da5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:52.037194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:52.059505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851669f-1985-437d-8648-51908a413292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:52.060127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851669f-1985-437d-8648-51908a413292 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:52.060407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851669f-1985-437d-8648-51908a413292 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:52.060676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851669f-1985-437d-8648-51908a413292 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:52.061383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851669f-1985-437d-8648-51908a413292 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:52.061691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851669f-1985-437d-8648-51908a413292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:52.063086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c851669f-1985-437d-8648-51908a413292 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:52.071078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 575/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:46:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:52.076952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df759e8a-d7c4-44bb-a3f1-19aa7635f46d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:52.173773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df759e8a-d7c4-44bb-a3f1-19aa7635f46d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:52.187056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:52.207575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b7faf-4ab1-425d-807b-1c86fbe074c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:52.208202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b7faf-4ab1-425d-807b-1c86fbe074c1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:52.208483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b7faf-4ab1-425d-807b-1c86fbe074c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:52.208755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b7faf-4ab1-425d-807b-1c86fbe074c1 None admin] PATH_INFO: `/v3/users/a7ccc11188b143008548dfa63294f187` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:52.221452 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b7faf-4ab1-425d-807b-1c86fbe074c1 None admin] RBAC: Authorizing `identity:get_user(user_id=a7ccc11188b143008548dfa63294f187)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:52.222932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b7faf-4ab1-425d-807b-1c86fbe074c1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:52.224537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 568/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:52 2022] GET /identity/v3/users/a7ccc11188b143008548dfa63294f187 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:52.231065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851669f-1985-437d-8648-51908a413292 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:52.328897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c851669f-1985-437d-8648-51908a413292 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:52.362685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a6f7d-cea0-43bc-9a87-694a8e1106df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:52.363296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a6f7d-cea0-43bc-9a87-694a8e1106df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:52.363577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a6f7d-cea0-43bc-9a87-694a8e1106df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:52.363849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a6f7d-cea0-43bc-9a87-694a8e1106df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:52.405656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:52.431259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a6f7d-cea0-43bc-9a87-694a8e1106df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:52.432649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a6f7d-cea0-43bc-9a87-694a8e1106df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:52.529526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 576/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:52.535364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b7faf-4ab1-425d-807b-1c86fbe074c1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:52.632320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b7faf-4ab1-425d-807b-1c86fbe074c1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:52.666858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804cfaa-9f40-4b4a-8478-cec50d5cb273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:52.667467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804cfaa-9f40-4b4a-8478-cec50d5cb273 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:52.667743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804cfaa-9f40-4b4a-8478-cec50d5cb273 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:52.668011 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804cfaa-9f40-4b4a-8478-cec50d5cb273 None admin] PATH_INFO: `/v3/projects/fc6772cdece24dc1bc5272f4e539ccd7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:52.669148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804cfaa-9f40-4b4a-8478-cec50d5cb273 None admin] RBAC: Authorizing `identity:get_project(project_id=fc6772cdece24dc1bc5272f4e539ccd7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:52.670583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804cfaa-9f40-4b4a-8478-cec50d5cb273 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:52.672210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 569/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:46:52 2022] GET /identity/v3/projects/fc6772cdece24dc1bc5272f4e539ccd7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:52.678194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a6f7d-cea0-43bc-9a87-694a8e1106df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:52.750911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:52.776377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a6f7d-cea0-43bc-9a87-694a8e1106df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:52.809244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4684e-de6a-4907-98ff-6b67099f96c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:52.809854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4684e-de6a-4907-98ff-6b67099f96c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:52.810137 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4684e-de6a-4907-98ff-6b67099f96c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:52.810411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4684e-de6a-4907-98ff-6b67099f96c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:52.811222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4684e-de6a-4907-98ff-6b67099f96c7 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=fc6772cdece24dc1bc5272f4e539ccd7, user.id=a7ccc11188b143008548dfa63294f187) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:52.811527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4684e-de6a-4907-98ff-6b67099f96c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:52.812955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4684e-de6a-4907-98ff-6b67099f96c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:52.818227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 577/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:52 2022] GET /identity/v3/role_assignments?user.id=a7ccc11188b143008548dfa63294f187&scope.project.id=fc6772cdece24dc1bc5272f4e539ccd7&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:54.865088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde2c9b-1c11-4be5-94a2-83a9b2af51c8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:54.865614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde2c9b-1c11-4be5-94a2-83a9b2af51c8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:54.865614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde2c9b-1c11-4be5-94a2-83a9b2af51c8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:54.867004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 570/1147] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:54.872979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b5675-aeaa-4b03-84ab-3df54c1d8462 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:54.873377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b5675-aeaa-4b03-84ab-3df54c1d8462 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:54.873534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b5675-aeaa-4b03-84ab-3df54c1d8462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:54.893258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-942b5675-aeaa-4b03-84ab-3df54c1d8462 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:54.894478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-942b5675-aeaa-4b03-84ab-3df54c1d8462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:54.960072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 578/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:54.982764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde2c9b-1c11-4be5-94a2-83a9b2af51c8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:55.024242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:55.082972 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde2c9b-1c11-4be5-94a2-83a9b2af51c8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:55.115332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47ea10-d815-49b1-b24a-489a1ee39322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:55.115903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47ea10-d815-49b1-b24a-489a1ee39322 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:55.116182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47ea10-d815-49b1-b24a-489a1ee39322 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:55.116455 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47ea10-d815-49b1-b24a-489a1ee39322 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:55.127070 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47ea10-d815-49b1-b24a-489a1ee39322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8b47ea10-d815-49b1-b24a-489a1ee39322 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:55.131901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:55.133484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:46:55.134398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 571/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:46:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:55.139766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b5675-aeaa-4b03-84ab-3df54c1d8462 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:55.204557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:55.236476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b5675-aeaa-4b03-84ab-3df54c1d8462 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:55.270106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd573d97-c242-4a68-adce-006776e6e173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:55.270740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd573d97-c242-4a68-adce-006776e6e173 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:55.271020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd573d97-c242-4a68-adce-006776e6e173 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:55.271292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd573d97-c242-4a68-adce-006776e6e173 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:55.272000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd573d97-c242-4a68-adce-006776e6e173 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:55.272301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd573d97-c242-4a68-adce-006776e6e173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:55.273941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd573d97-c242-4a68-adce-006776e6e173 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:55.281784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 579/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:46:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:55.287500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47ea10-d815-49b1-b24a-489a1ee39322 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:55.382085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:55.386203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47ea10-d815-49b1-b24a-489a1ee39322 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:55.419765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf41a0b-619c-4d6f-b11e-ccafb3fb276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:55.420389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf41a0b-619c-4d6f-b11e-ccafb3fb276e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:55.420666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf41a0b-619c-4d6f-b11e-ccafb3fb276e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:55.420932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf41a0b-619c-4d6f-b11e-ccafb3fb276e None admin] PATH_INFO: `/v3/users/a7ccc11188b143008548dfa63294f187` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:55.422073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf41a0b-619c-4d6f-b11e-ccafb3fb276e None admin] RBAC: Authorizing `identity:get_user(user_id=a7ccc11188b143008548dfa63294f187)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:55.423526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf41a0b-619c-4d6f-b11e-ccafb3fb276e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:55.425103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 572/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:55 2022] GET /identity/v3/users/a7ccc11188b143008548dfa63294f187 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:55.431675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd573d97-c242-4a68-adce-006776e6e173 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:55.528888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd573d97-c242-4a68-adce-006776e6e173 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:55.556400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:55.563056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710f251-a46f-4c6b-a627-37893bfe741c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:55.563686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710f251-a46f-4c6b-a627-37893bfe741c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:55.563962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710f251-a46f-4c6b-a627-37893bfe741c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:55.564233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710f251-a46f-4c6b-a627-37893bfe741c None admin] PATH_INFO: `/v3/projects/fc6772cdece24dc1bc5272f4e539ccd7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:55.565368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710f251-a46f-4c6b-a627-37893bfe741c None admin] RBAC: Authorizing `identity:get_project(project_id=fc6772cdece24dc1bc5272f4e539ccd7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:55.566810 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710f251-a46f-4c6b-a627-37893bfe741c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:55.568437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 580/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:46:55 2022] GET /identity/v3/projects/fc6772cdece24dc1bc5272f4e539ccd7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:46:55.575073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf41a0b-619c-4d6f-b11e-ccafb3fb276e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:55.671028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf41a0b-619c-4d6f-b11e-ccafb3fb276e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:55.703287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04133c4-a6e9-4aa3-aa2a-882e97e237f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:55.703932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04133c4-a6e9-4aa3-aa2a-882e97e237f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:55.704209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04133c4-a6e9-4aa3-aa2a-882e97e237f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:55.704479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04133c4-a6e9-4aa3-aa2a-882e97e237f9 None admin] PATH_INFO: `/v3/projects/fc6772cdece24dc1bc5272f4e539ccd7/users/a7ccc11188b143008548dfa63294f187/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:55.706387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04133c4-a6e9-4aa3-aa2a-882e97e237f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc6772cdece24dc1bc5272f4e539ccd7, user_id=a7ccc11188b143008548dfa63294f187, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:55.707832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04133c4-a6e9-4aa3-aa2a-882e97e237f9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:55.721648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 573/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:46:55 2022] PUT /identity/v3/projects/fc6772cdece24dc1bc5272f4e539ccd7/users/a7ccc11188b143008548dfa63294f187/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:46:57.696511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111fb79-dfab-4566-8da0-93b1a6bbfc80 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:57.697069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111fb79-dfab-4566-8da0-93b1a6bbfc80 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:57.697069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111fb79-dfab-4566-8da0-93b1a6bbfc80 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:57.698390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 581/1154] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:46:57.703870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd77ec4-82f6-4588-ae7b-3ef3769a1c5b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:57.704295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd77ec4-82f6-4588-ae7b-3ef3769a1c5b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:57.704469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd77ec4-82f6-4588-ae7b-3ef3769a1c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:57.722277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2dd77ec4-82f6-4588-ae7b-3ef3769a1c5b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:46:57.723476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2dd77ec4-82f6-4588-ae7b-3ef3769a1c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:46:57.744239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:57.789725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 574/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:46:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:46:57.812598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111fb79-dfab-4566-8da0-93b1a6bbfc80 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:57.914580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111fb79-dfab-4566-8da0-93b1a6bbfc80 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:57.947806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968f1f-8096-4e8a-b202-b98cbd505acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:57.948364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968f1f-8096-4e8a-b202-b98cbd505acb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:57.948641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968f1f-8096-4e8a-b202-b98cbd505acb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:57.948912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968f1f-8096-4e8a-b202-b98cbd505acb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:57.959787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f968f1f-8096-4e8a-b202-b98cbd505acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2f968f1f-8096-4e8a-b202-b98cbd505acb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:46:57.964596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:46:57.970692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:46:57.971836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 582/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:46:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:46:57.972714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd77ec4-82f6-4588-ae7b-3ef3769a1c5b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:58.067647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd77ec4-82f6-4588-ae7b-3ef3769a1c5b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:58.101339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11655c02-789a-4338-9668-a095c89e1284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:58.101950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11655c02-789a-4338-9668-a095c89e1284 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:58.102229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11655c02-789a-4338-9668-a095c89e1284 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:58.102532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11655c02-789a-4338-9668-a095c89e1284 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:58.103248 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11655c02-789a-4338-9668-a095c89e1284 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:58.103550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11655c02-789a-4338-9668-a095c89e1284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:58.104908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11655c02-789a-4338-9668-a095c89e1284 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:58.112680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 575/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:46:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:58.119477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968f1f-8096-4e8a-b202-b98cbd505acb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:58.176468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:58.218019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968f1f-8096-4e8a-b202-b98cbd505acb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:58.250552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636ec2d-c2bc-430e-98f2-ec01e7550f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:58.251173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ec2d-c2bc-430e-98f2-ec01e7550f8c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:58.251450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ec2d-c2bc-430e-98f2-ec01e7550f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:58.251719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ec2d-c2bc-430e-98f2-ec01e7550f8c None admin] PATH_INFO: `/v3/users/a7ccc11188b143008548dfa63294f187` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:58.252855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636ec2d-c2bc-430e-98f2-ec01e7550f8c None admin] RBAC: Authorizing `identity:get_user(user_id=a7ccc11188b143008548dfa63294f187)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:58.254272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636ec2d-c2bc-430e-98f2-ec01e7550f8c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:58.255876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 583/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:46:58 2022] GET /identity/v3/users/a7ccc11188b143008548dfa63294f187 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:58.262610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11655c02-789a-4338-9668-a095c89e1284 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:58.282244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:58.362098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11655c02-789a-4338-9668-a095c89e1284 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:58.395288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebba4e-73f7-423e-800d-423d241b367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:58.395891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebba4e-73f7-423e-800d-423d241b367a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:58.396171 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebba4e-73f7-423e-800d-423d241b367a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:58.396443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebba4e-73f7-423e-800d-423d241b367a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:58.464571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebba4e-73f7-423e-800d-423d241b367a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:58.465997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebba4e-73f7-423e-800d-423d241b367a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:58.483792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:58.564053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 576/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:46:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:46:58.569939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636ec2d-c2bc-430e-98f2-ec01e7550f8c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:58.667824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636ec2d-c2bc-430e-98f2-ec01e7550f8c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:58.691598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:58.700806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577640c-d6d9-44b1-9568-f7bdb72b35a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:58.701471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577640c-d6d9-44b1-9568-f7bdb72b35a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:58.701714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577640c-d6d9-44b1-9568-f7bdb72b35a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:58.701978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577640c-d6d9-44b1-9568-f7bdb72b35a2 None admin] PATH_INFO: `/v3/projects/fc6772cdece24dc1bc5272f4e539ccd7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:58.703178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577640c-d6d9-44b1-9568-f7bdb72b35a2 None admin] RBAC: Authorizing `identity:get_project(project_id=fc6772cdece24dc1bc5272f4e539ccd7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:58.704595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577640c-d6d9-44b1-9568-f7bdb72b35a2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:58.706231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 584/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:46:58 2022] GET /identity/v3/projects/fc6772cdece24dc1bc5272f4e539ccd7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:46:58.713119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebba4e-73f7-423e-800d-423d241b367a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:46:58.810208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebba4e-73f7-423e-800d-423d241b367a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:46:58.827214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:46:58.844433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a01670-e67a-4a7f-ac4b-23369c8f27fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:46:58.845032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a01670-e67a-4a7f-ac4b-23369c8f27fe None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:46:58.845310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a01670-e67a-4a7f-ac4b-23369c8f27fe None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:46:58.845582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a01670-e67a-4a7f-ac4b-23369c8f27fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:46:58.846358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a01670-e67a-4a7f-ac4b-23369c8f27fe None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=fc6772cdece24dc1bc5272f4e539ccd7, user.id=a7ccc11188b143008548dfa63294f187) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:46:58.846694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a01670-e67a-4a7f-ac4b-23369c8f27fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:46:58.848106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a01670-e67a-4a7f-ac4b-23369c8f27fe None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:46:58.854414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 577/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:46:58 2022] GET /identity/v3/role_assignments?user.id=a7ccc11188b143008548dfa63294f187&scope.project.id=fc6772cdece24dc1bc5272f4e539ccd7&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:00.853621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304f781-901e-4d81-95b4-58e1123a82c7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:00.854170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304f781-901e-4d81-95b4-58e1123a82c7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:00.854170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304f781-901e-4d81-95b4-58e1123a82c7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:00.855548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 585/1162] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:47:00.860664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df46e2-9f8e-4871-926a-bdc474467559 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:00.861067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df46e2-9f8e-4871-926a-bdc474467559 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:00.861205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df46e2-9f8e-4871-926a-bdc474467559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:00.879319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-48df46e2-9f8e-4871-926a-bdc474467559 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:47:00.880494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-48df46e2-9f8e-4871-926a-bdc474467559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:47:00.945773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 578/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:47:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:47:00.968032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304f781-901e-4d81-95b4-58e1123a82c7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:01.071136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304f781-901e-4d81-95b4-58e1123a82c7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:01.105935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe22d5-e8ed-4c88-9b2e-2d8011a58245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:01.106511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe22d5-e8ed-4c88-9b2e-2d8011a58245 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:01.106793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe22d5-e8ed-4c88-9b2e-2d8011a58245 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:01.107064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe22d5-e8ed-4c88-9b2e-2d8011a58245 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:01.165808 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:01.174133 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe22d5-e8ed-4c88-9b2e-2d8011a58245 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:01.175601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe22d5-e8ed-4c88-9b2e-2d8011a58245 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:01.270216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 586/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:47:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:47:01.277173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df46e2-9f8e-4871-926a-bdc474467559 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:01.371407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df46e2-9f8e-4871-926a-bdc474467559 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:01.384329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:01.405258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcbcec-7c85-46bc-b468-7be0c5283329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:01.405880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcbcec-7c85-46bc-b468-7be0c5283329 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:01.406154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcbcec-7c85-46bc-b468-7be0c5283329 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:01.406419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcbcec-7c85-46bc-b468-7be0c5283329 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:01.412824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcbcec-7c85-46bc-b468-7be0c5283329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:01.414309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcbcec-7c85-46bc-b468-7be0c5283329 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c1dcbcec-7c85-46bc-b468-7be0c5283329 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:47:01.425284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:47:01.426593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:47:01.427741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 579/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:47:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:47:01.432573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe22d5-e8ed-4c88-9b2e-2d8011a58245 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:01.523191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:01.530265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe22d5-e8ed-4c88-9b2e-2d8011a58245 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:01.562370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbdf78-08f7-48e1-bea9-8c63c9555cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:01.563001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbdf78-08f7-48e1-bea9-8c63c9555cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:01.563276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbdf78-08f7-48e1-bea9-8c63c9555cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:01.563544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbdf78-08f7-48e1-bea9-8c63c9555cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:01.564217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbdf78-08f7-48e1-bea9-8c63c9555cb4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:47:01.564515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbdf78-08f7-48e1-bea9-8c63c9555cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:01.565851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbdf78-08f7-48e1-bea9-8c63c9555cb4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:01.573297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 587/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Jul 30 04:47:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:03.531912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2884db1-e802-4cfe-93a3-17c638d0373d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:03.532460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2884db1-e802-4cfe-93a3-17c638d0373d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:03.532460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2884db1-e802-4cfe-93a3-17c638d0373d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:03.533819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 580/1167] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:47:03.539540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babdc75-603a-4823-b53e-ef95486308e2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:03.539950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babdc75-603a-4823-b53e-ef95486308e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:03.540089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babdc75-603a-4823-b53e-ef95486308e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:03.559203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4babdc75-603a-4823-b53e-ef95486308e2 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:47:03.560386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4babdc75-603a-4823-b53e-ef95486308e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:47:03.624430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 588/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:47:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:47:03.646491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2884db1-e802-4cfe-93a3-17c638d0373d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:03.746578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2884db1-e802-4cfe-93a3-17c638d0373d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:03.779325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c788e0-9b87-420d-a959-9448a619ef52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:03.779929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c788e0-9b87-420d-a959-9448a619ef52 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:03.780206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c788e0-9b87-420d-a959-9448a619ef52 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:03.780476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c788e0-9b87-420d-a959-9448a619ef52 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:03.781169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c788e0-9b87-420d-a959-9448a619ef52 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:03.782605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c788e0-9b87-420d-a959-9448a619ef52 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:03.798312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 581/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Sat Jul 30 04:47:03 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:47:05.854774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518f5b6-a753-446e-8374-e6adcb8d9afb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:05.855347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518f5b6-a753-446e-8374-e6adcb8d9afb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:05.855347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518f5b6-a753-446e-8374-e6adcb8d9afb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:05.856651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 589/1170] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:47:05.861731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94ed16-184e-449a-88e8-259eaca7a8bd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:05.862147 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94ed16-184e-449a-88e8-259eaca7a8bd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:05.862286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94ed16-184e-449a-88e8-259eaca7a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:05.880927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9b94ed16-184e-449a-88e8-259eaca7a8bd None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:47:05.882378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9b94ed16-184e-449a-88e8-259eaca7a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:47:05.907595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:05.948812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 582/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:47:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:47:05.971917 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518f5b6-a753-446e-8374-e6adcb8d9afb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:06.060587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:06.073365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518f5b6-a753-446e-8374-e6adcb8d9afb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:06.106353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa6749-9303-47ae-a35c-d7277410c12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:06.106949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa6749-9303-47ae-a35c-d7277410c12f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:06.107222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa6749-9303-47ae-a35c-d7277410c12f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:06.107489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa6749-9303-47ae-a35c-d7277410c12f None admin] PATH_INFO: `/v3/domains/d4e095d576ca4672997cf9233e67af3f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:06.118105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa6749-9303-47ae-a35c-d7277410c12f None admin] RBAC: Authorizing `identity:get_domain(domain_id=d4e095d576ca4672997cf9233e67af3f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:06.119570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa6749-9303-47ae-a35c-d7277410c12f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:06.121154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 590/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Sat Jul 30 04:47:05 2022] GET /identity/v3/domains/d4e095d576ca4672997cf9233e67af3f => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:06.127026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94ed16-184e-449a-88e8-259eaca7a8bd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:06.222135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94ed16-184e-449a-88e8-259eaca7a8bd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:06.254565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf185a-7df2-4bb0-924a-9a5ad370bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:06.255155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf185a-7df2-4bb0-924a-9a5ad370bf30 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:06.255426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf185a-7df2-4bb0-924a-9a5ad370bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:06.255693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf185a-7df2-4bb0-924a-9a5ad370bf30 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:06.256390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf185a-7df2-4bb0-924a-9a5ad370bf30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:06.257981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf185a-7df2-4bb0-924a-9a5ad370bf30 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:06.283958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 583/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:47:06 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:47:08.327174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49a4e2-bc99-47d3-ab47-a84d712bf683 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:08.327674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49a4e2-bc99-47d3-ab47-a84d712bf683 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:08.327674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49a4e2-bc99-47d3-ab47-a84d712bf683 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:08.328972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 591/1174] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:47:08.334026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468ce19-d99c-4a8a-aec1-9e74619d8834 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:08.334594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468ce19-d99c-4a8a-aec1-9e74619d8834 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:08.334753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468ce19-d99c-4a8a-aec1-9e74619d8834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:08.352858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a468ce19-d99c-4a8a-aec1-9e74619d8834 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:47:08.354064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a468ce19-d99c-4a8a-aec1-9e74619d8834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:47:08.363024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:08.420486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 584/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:47:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:47:08.444087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49a4e2-bc99-47d3-ab47-a84d712bf683 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:08.544375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49a4e2-bc99-47d3-ab47-a84d712bf683 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:08.576477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d442e-d462-434d-8082-43a5d6784a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:08.577103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d442e-d462-434d-8082-43a5d6784a7c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:08.577430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d442e-d462-434d-8082-43a5d6784a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:08.577655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d442e-d462-434d-8082-43a5d6784a7c None admin] PATH_INFO: `/v3/domains/d4e095d576ca4672997cf9233e67af3f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:08.578819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d442e-d462-434d-8082-43a5d6784a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=d4e095d576ca4672997cf9233e67af3f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:08.580235 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d442e-d462-434d-8082-43a5d6784a7c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:08.581792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 592/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Sat Jul 30 04:47:08 2022] GET /identity/v3/domains/d4e095d576ca4672997cf9233e67af3f => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:08.587896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468ce19-d99c-4a8a-aec1-9e74619d8834 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:08.684549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468ce19-d99c-4a8a-aec1-9e74619d8834 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:08.717270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edff5ea-66f6-4b1a-ac7f-b4ffef94b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:08.717871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edff5ea-66f6-4b1a-ac7f-b4ffef94b362 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:08.718145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edff5ea-66f6-4b1a-ac7f-b4ffef94b362 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:08.718413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edff5ea-66f6-4b1a-ac7f-b4ffef94b362 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:08.719108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edff5ea-66f6-4b1a-ac7f-b4ffef94b362 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:08.720520 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edff5ea-66f6-4b1a-ac7f-b4ffef94b362 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:08.736910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:08.743655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 585/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:47:08 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:47:10.793813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0906f2-6b4c-4f64-a9fa-ce5e7fe1cb09 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:10.794365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0906f2-6b4c-4f64-a9fa-ce5e7fe1cb09 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:10.794365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0906f2-6b4c-4f64-a9fa-ce5e7fe1cb09 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:10.795873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 593/1178] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:47:10.800879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de13f31-93a3-4bb3-a24a-4a96708f7970 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:10.801307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de13f31-93a3-4bb3-a24a-4a96708f7970 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:10.801440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de13f31-93a3-4bb3-a24a-4a96708f7970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:10.821212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7de13f31-93a3-4bb3-a24a-4a96708f7970 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:47:10.822404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7de13f31-93a3-4bb3-a24a-4a96708f7970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:47:10.888707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 586/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:47:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:47:10.911811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0906f2-6b4c-4f64-a9fa-ce5e7fe1cb09 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:10.930516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:11.014953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0906f2-6b4c-4f64-a9fa-ce5e7fe1cb09 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:11.049713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b880-3edc-4503-8ba3-c67f7cc5d730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:11.050295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b880-3edc-4503-8ba3-c67f7cc5d730 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:11.050611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b880-3edc-4503-8ba3-c67f7cc5d730 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:11.050889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827b880-3edc-4503-8ba3-c67f7cc5d730 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:11.062008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827b880-3edc-4503-8ba3-c67f7cc5d730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5827b880-3edc-4503-8ba3-c67f7cc5d730 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:47:11.066912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:47:11.067910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:47:11.068781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 594/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:47:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:47:11.074378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de13f31-93a3-4bb3-a24a-4a96708f7970 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:11.170589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de13f31-93a3-4bb3-a24a-4a96708f7970 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:11.199007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:11.205194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c685518-9f73-4189-b4d4-3b1eb98cb194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:11.205811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c685518-9f73-4189-b4d4-3b1eb98cb194 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:11.206088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c685518-9f73-4189-b4d4-3b1eb98cb194 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:11.206358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c685518-9f73-4189-b4d4-3b1eb98cb194 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:11.207101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c685518-9f73-4189-b4d4-3b1eb98cb194 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:47:11.207400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c685518-9f73-4189-b4d4-3b1eb98cb194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:11.208761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c685518-9f73-4189-b4d4-3b1eb98cb194 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:11.217432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 587/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:47:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:11.223135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b880-3edc-4503-8ba3-c67f7cc5d730 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:11.298854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:11.322556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827b880-3edc-4503-8ba3-c67f7cc5d730 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:11.355509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333489e-81eb-4762-9806-81a35932d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:11.356138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333489e-81eb-4762-9806-81a35932d88a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:11.356419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333489e-81eb-4762-9806-81a35932d88a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:11.356694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333489e-81eb-4762-9806-81a35932d88a None admin] PATH_INFO: `/v3/users/f516f8cf14fe40c2a78b86dd88ca2cdc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:11.369593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333489e-81eb-4762-9806-81a35932d88a None admin] RBAC: Authorizing `identity:get_user(user_id=f516f8cf14fe40c2a78b86dd88ca2cdc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:11.371138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333489e-81eb-4762-9806-81a35932d88a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:11.372745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 595/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:47:11 2022] GET /identity/v3/users/f516f8cf14fe40c2a78b86dd88ca2cdc => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:11.378430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c685518-9f73-4189-b4d4-3b1eb98cb194 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:11.477947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c685518-9f73-4189-b4d4-3b1eb98cb194 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:11.511128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db97c6-2116-4735-bfa8-6d24d27d51aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:11.511731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db97c6-2116-4735-bfa8-6d24d27d51aa None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:11.512010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db97c6-2116-4735-bfa8-6d24d27d51aa None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:11.512279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db97c6-2116-4735-bfa8-6d24d27d51aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:11.569943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:11.579331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db97c6-2116-4735-bfa8-6d24d27d51aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:11.580700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db97c6-2116-4735-bfa8-6d24d27d51aa None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:11.676227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 588/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:47:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:47:11.682625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333489e-81eb-4762-9806-81a35932d88a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:11.782861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333489e-81eb-4762-9806-81a35932d88a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:11.815985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906444ea-4280-41e5-9051-d989e9b31941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:11.816610 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906444ea-4280-41e5-9051-d989e9b31941 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:11.816885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906444ea-4280-41e5-9051-d989e9b31941 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:11.817153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906444ea-4280-41e5-9051-d989e9b31941 None admin] PATH_INFO: `/v3/projects/ef1c6b055c28488ea6f20c44b95b8289` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:11.818975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906444ea-4280-41e5-9051-d989e9b31941 None admin] RBAC: Authorizing `identity:get_project(project_id=ef1c6b055c28488ea6f20c44b95b8289)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:11.820388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906444ea-4280-41e5-9051-d989e9b31941 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:11.822018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 596/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:47:11 2022] GET /identity/v3/projects/ef1c6b055c28488ea6f20c44b95b8289 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:11.828059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db97c6-2116-4735-bfa8-6d24d27d51aa None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:11.916683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:11.925344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db97c6-2116-4735-bfa8-6d24d27d51aa None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:11.958490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed1f5f-af52-4b24-a6f1-7b6e71307e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:11.959814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed1f5f-af52-4b24-a6f1-7b6e71307e09 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:11.960084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed1f5f-af52-4b24-a6f1-7b6e71307e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:11.960351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed1f5f-af52-4b24-a6f1-7b6e71307e09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:11.961119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed1f5f-af52-4b24-a6f1-7b6e71307e09 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=ef1c6b055c28488ea6f20c44b95b8289, user.id=f516f8cf14fe40c2a78b86dd88ca2cdc) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:47:11.961419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed1f5f-af52-4b24-a6f1-7b6e71307e09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:11.962851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed1f5f-af52-4b24-a6f1-7b6e71307e09 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:11.968103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 589/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:47:11 2022] GET /identity/v3/role_assignments?user.id=f516f8cf14fe40c2a78b86dd88ca2cdc&scope.project.id=ef1c6b055c28488ea6f20c44b95b8289&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:13.930736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca05cd-912d-48e2-834c-202abbffca22 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:13.931320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca05cd-912d-48e2-834c-202abbffca22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:13.931320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca05cd-912d-48e2-834c-202abbffca22 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:13.932638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 597/1186] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:47:13.937830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221426b-885f-40a2-86a6-6fc2e1263cda None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:13.938267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221426b-885f-40a2-86a6-6fc2e1263cda None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:13.938374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221426b-885f-40a2-86a6-6fc2e1263cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:13.965664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d221426b-885f-40a2-86a6-6fc2e1263cda None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:47:13.966895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d221426b-885f-40a2-86a6-6fc2e1263cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:47:14.033873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 590/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:47:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:47:14.057088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca05cd-912d-48e2-834c-202abbffca22 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:14.059942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:14.158096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca05cd-912d-48e2-834c-202abbffca22 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:14.191091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59552a50-866c-4af3-b4b0-83cee8727331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:14.191662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59552a50-866c-4af3-b4b0-83cee8727331 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:14.191944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59552a50-866c-4af3-b4b0-83cee8727331 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:14.192220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59552a50-866c-4af3-b4b0-83cee8727331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:14.202696 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59552a50-866c-4af3-b4b0-83cee8727331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-59552a50-866c-4af3-b4b0-83cee8727331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:47:14.207542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:47:14.209250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:47:14.210176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 598/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:47:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:47:14.215261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221426b-885f-40a2-86a6-6fc2e1263cda None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:14.298238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:14.312925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221426b-885f-40a2-86a6-6fc2e1263cda None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:14.347133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90563dd-5d26-48fe-a4fe-a1c5294ec1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:14.347742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90563dd-5d26-48fe-a4fe-a1c5294ec1ed None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:14.348019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90563dd-5d26-48fe-a4fe-a1c5294ec1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:14.348290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90563dd-5d26-48fe-a4fe-a1c5294ec1ed None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:14.348996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90563dd-5d26-48fe-a4fe-a1c5294ec1ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:47:14.349335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90563dd-5d26-48fe-a4fe-a1c5294ec1ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:14.351039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90563dd-5d26-48fe-a4fe-a1c5294ec1ed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:14.358811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 591/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:47:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:14.364552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59552a50-866c-4af3-b4b0-83cee8727331 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:14.426085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:14.465351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59552a50-866c-4af3-b4b0-83cee8727331 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:14.499180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714073d2-fe59-41c0-ab1f-c161be5604d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:14.499809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714073d2-fe59-41c0-ab1f-c161be5604d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:14.500091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714073d2-fe59-41c0-ab1f-c161be5604d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:14.500367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714073d2-fe59-41c0-ab1f-c161be5604d2 None admin] PATH_INFO: `/v3/users/f516f8cf14fe40c2a78b86dd88ca2cdc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:14.501527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714073d2-fe59-41c0-ab1f-c161be5604d2 None admin] RBAC: Authorizing `identity:get_user(user_id=f516f8cf14fe40c2a78b86dd88ca2cdc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:14.502977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714073d2-fe59-41c0-ab1f-c161be5604d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:14.504552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 599/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:47:14 2022] GET /identity/v3/users/f516f8cf14fe40c2a78b86dd88ca2cdc => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:14.510901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90563dd-5d26-48fe-a4fe-a1c5294ec1ed None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:14.607929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90563dd-5d26-48fe-a4fe-a1c5294ec1ed None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:14.640561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6070a-f50b-447c-ac47-154fd4a9ed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:14.641180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6070a-f50b-447c-ac47-154fd4a9ed2c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:14.641454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6070a-f50b-447c-ac47-154fd4a9ed2c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:14.641726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6070a-f50b-447c-ac47-154fd4a9ed2c None admin] PATH_INFO: `/v3/projects/ef1c6b055c28488ea6f20c44b95b8289` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:14.642880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6070a-f50b-447c-ac47-154fd4a9ed2c None admin] RBAC: Authorizing `identity:get_project(project_id=ef1c6b055c28488ea6f20c44b95b8289)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:14.644285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6070a-f50b-447c-ac47-154fd4a9ed2c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:14.645904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 592/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:47:14 2022] GET /identity/v3/projects/ef1c6b055c28488ea6f20c44b95b8289 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:14.651842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714073d2-fe59-41c0-ab1f-c161be5604d2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:14.749610 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714073d2-fe59-41c0-ab1f-c161be5604d2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:14.777915 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:14.784060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e87bf-3d74-4289-9f75-84a3de9bc431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:14.784729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e87bf-3d74-4289-9f75-84a3de9bc431 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:14.785011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e87bf-3d74-4289-9f75-84a3de9bc431 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:14.785287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e87bf-3d74-4289-9f75-84a3de9bc431 None admin] PATH_INFO: `/v3/projects/ef1c6b055c28488ea6f20c44b95b8289/users/f516f8cf14fe40c2a78b86dd88ca2cdc/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:14.788033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e87bf-3d74-4289-9f75-84a3de9bc431 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef1c6b055c28488ea6f20c44b95b8289, user_id=f516f8cf14fe40c2a78b86dd88ca2cdc, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:14.789452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e87bf-3d74-4289-9f75-84a3de9bc431 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:14.801683 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 600/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:47:14 2022] PUT /identity/v3/projects/ef1c6b055c28488ea6f20c44b95b8289/users/f516f8cf14fe40c2a78b86dd88ca2cdc/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:47:16.772753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf9f33-2ec9-4ea4-bd7d-cc871315f281 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:16.773310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf9f33-2ec9-4ea4-bd7d-cc871315f281 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:16.773310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf9f33-2ec9-4ea4-bd7d-cc871315f281 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:16.774655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 593/1193] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:47:16.779715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c126991-dc45-4082-bc49-1c2c40c45bf6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:16.780099 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c126991-dc45-4082-bc49-1c2c40c45bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:16.780267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c126991-dc45-4082-bc49-1c2c40c45bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:16.798227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8c126991-dc45-4082-bc49-1c2c40c45bf6 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:47:16.799481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c126991-dc45-4082-bc49-1c2c40c45bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:47:16.864476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 601/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:47:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:47:16.888768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf9f33-2ec9-4ea4-bd7d-cc871315f281 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:16.908193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:16.991645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf9f33-2ec9-4ea4-bd7d-cc871315f281 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:17.026340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bfa90-4001-463d-b61f-0623a0cff058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:17.026958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bfa90-4001-463d-b61f-0623a0cff058 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:17.027230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bfa90-4001-463d-b61f-0623a0cff058 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:17.027492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bfa90-4001-463d-b61f-0623a0cff058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:17.037952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bfa90-4001-463d-b61f-0623a0cff058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0d5bfa90-4001-463d-b61f-0623a0cff058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:47:17.042773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:47:17.043949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:47:17.045273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 594/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:47:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:47:17.050149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c126991-dc45-4082-bc49-1c2c40c45bf6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:17.150180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c126991-dc45-4082-bc49-1c2c40c45bf6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:17.170289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:17.184945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de94ce-a5cf-4804-ae90-068838426ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:17.185561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de94ce-a5cf-4804-ae90-068838426ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:17.185847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de94ce-a5cf-4804-ae90-068838426ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:17.186121 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de94ce-a5cf-4804-ae90-068838426ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:17.186878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de94ce-a5cf-4804-ae90-068838426ac3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:47:17.187181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de94ce-a5cf-4804-ae90-068838426ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:17.188554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de94ce-a5cf-4804-ae90-068838426ac3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:17.197077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 602/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:47:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:17.202942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bfa90-4001-463d-b61f-0623a0cff058 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:17.278799 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:17.301387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bfa90-4001-463d-b61f-0623a0cff058 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:17.334341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a8115-16d5-41c5-9920-cda278c6cb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:17.335009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a8115-16d5-41c5-9920-cda278c6cb07 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:17.335289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a8115-16d5-41c5-9920-cda278c6cb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:17.335575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a8115-16d5-41c5-9920-cda278c6cb07 None admin] PATH_INFO: `/v3/users/f516f8cf14fe40c2a78b86dd88ca2cdc` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:17.336722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a8115-16d5-41c5-9920-cda278c6cb07 None admin] RBAC: Authorizing `identity:get_user(user_id=f516f8cf14fe40c2a78b86dd88ca2cdc)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:17.338145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a8115-16d5-41c5-9920-cda278c6cb07 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:17.339763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 595/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Jul 30 04:47:17 2022] GET /identity/v3/users/f516f8cf14fe40c2a78b86dd88ca2cdc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:17.345539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de94ce-a5cf-4804-ae90-068838426ac3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:17.445815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de94ce-a5cf-4804-ae90-068838426ac3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:17.479700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e496c-f1e6-4501-bd60-89f2960e3362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:17.480304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e496c-f1e6-4501-bd60-89f2960e3362 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:17.480586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e496c-f1e6-4501-bd60-89f2960e3362 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:17.480860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e496c-f1e6-4501-bd60-89f2960e3362 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:17.528996 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:17.547356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e496c-f1e6-4501-bd60-89f2960e3362 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:17.548804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e496c-f1e6-4501-bd60-89f2960e3362 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:17.645259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 603/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:47:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:47:17.651406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a8115-16d5-41c5-9920-cda278c6cb07 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:17.746917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a8115-16d5-41c5-9920-cda278c6cb07 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:17.779259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccc890-5d7b-4489-9bcb-f2e40e437314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:17.779838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccc890-5d7b-4489-9bcb-f2e40e437314 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:17.780086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccc890-5d7b-4489-9bcb-f2e40e437314 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:17.780354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccc890-5d7b-4489-9bcb-f2e40e437314 None admin] PATH_INFO: `/v3/projects/ef1c6b055c28488ea6f20c44b95b8289` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:17.781483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccc890-5d7b-4489-9bcb-f2e40e437314 None admin] RBAC: Authorizing `identity:get_project(project_id=ef1c6b055c28488ea6f20c44b95b8289)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:17.782923 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccc890-5d7b-4489-9bcb-f2e40e437314 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:17.784542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 596/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Jul 30 04:47:17 2022] GET /identity/v3/projects/ef1c6b055c28488ea6f20c44b95b8289 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:17.791423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e496c-f1e6-4501-bd60-89f2960e3362 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:47:17.871449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:17.890334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e496c-f1e6-4501-bd60-89f2960e3362 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:47:17.924172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b13a72-5051-42ed-95a0-4cb5bebca193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:47:17.924781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b13a72-5051-42ed-95a0-4cb5bebca193 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:47:17.925063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b13a72-5051-42ed-95a0-4cb5bebca193 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:47:17.925337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b13a72-5051-42ed-95a0-4cb5bebca193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:47:17.926118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b13a72-5051-42ed-95a0-4cb5bebca193 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=ef1c6b055c28488ea6f20c44b95b8289, user.id=f516f8cf14fe40c2a78b86dd88ca2cdc) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:47:17.926426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b13a72-5051-42ed-95a0-4cb5bebca193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:47:17.927964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b13a72-5051-42ed-95a0-4cb5bebca193 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:47:17.933626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 604/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:47:17 2022] GET /identity/v3/role_assignments?user.id=f516f8cf14fe40c2a78b86dd88ca2cdc&scope.project.id=ef1c6b055c28488ea6f20c44b95b8289&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:47:27.775135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:47:27.931094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=27 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:13.229440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c1e62-c611-465a-9bdd-4c00cadcffd0 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:13.230027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c1e62-c611-465a-9bdd-4c00cadcffd0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:13.230362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c1e62-c611-465a-9bdd-4c00cadcffd0 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:13.234581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 597/1201] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:13.239685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55ca72-f589-4421-a91c-6a7ecc7eba83 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:13.240143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55ca72-f589-4421-a91c-6a7ecc7eba83 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:13.240325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55ca72-f589-4421-a91c-6a7ecc7eba83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:13.267939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0d55ca72-f589-4421-a91c-6a7ecc7eba83 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:13.269401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d55ca72-f589-4421-a91c-6a7ecc7eba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:13.354822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 605/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:13.382387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c1e62-c611-465a-9bdd-4c00cadcffd0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:13.518118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c1e62-c611-465a-9bdd-4c00cadcffd0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:13.562616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9a8a4-43c0-4f07-a0f9-b324fee75c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:13.563335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9a8a4-43c0-4f07-a0f9-b324fee75c08 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:13.563659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9a8a4-43c0-4f07-a0f9-b324fee75c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:13.563987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9a8a4-43c0-4f07-a0f9-b324fee75c08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:13.581464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9a8a4-43c0-4f07-a0f9-b324fee75c08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:13.583369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9a8a4-43c0-4f07-a0f9-b324fee75c08 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ddf9a8a4-43c0-4f07-a0f9-b324fee75c08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:13.603381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:13.604517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:13.605568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:13.605568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:13.605568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:13.605568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:13.611213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 598/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:48:13.612049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55ca72-f589-4421-a91c-6a7ecc7eba83 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:13.752976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55ca72-f589-4421-a91c-6a7ecc7eba83 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:13.790719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:13.797617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e517434-2280-4b47-bba6-e63bf0127c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:13.798627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517434-2280-4b47-bba6-e63bf0127c35 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:13.798827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517434-2280-4b47-bba6-e63bf0127c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:13.799309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517434-2280-4b47-bba6-e63bf0127c35 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:13.800069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e517434-2280-4b47-bba6-e63bf0127c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:13.800377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e517434-2280-4b47-bba6-e63bf0127c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:13.801798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e517434-2280-4b47-bba6-e63bf0127c35 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:13.815674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 606/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:48:13.822704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9a8a4-43c0-4f07-a0f9-b324fee75c08 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:13.890332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:13.960292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9a8a4-43c0-4f07-a0f9-b324fee75c08 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:14.014232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1d635-6e89-4222-9b8f-f828983915e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:14.015819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1d635-6e89-4222-9b8f-f828983915e5 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:14.016928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1d635-6e89-4222-9b8f-f828983915e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:14.022078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1d635-6e89-4222-9b8f-f828983915e5 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:14.022078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1d635-6e89-4222-9b8f-f828983915e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:14.022317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1d635-6e89-4222-9b8f-f828983915e5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:14.048143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 599/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Jul 30 04:48:13 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:48:16.239203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce7a90-b858-4451-b5d1-ead6f0e8ad3a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:16.239787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce7a90-b858-4451-b5d1-ead6f0e8ad3a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:16.239787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce7a90-b858-4451-b5d1-ead6f0e8ad3a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:16.241299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 607/1206] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:16.247139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a107a8-e79c-4fa9-97ae-b8fab4bdfb62 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:16.247597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a107a8-e79c-4fa9-97ae-b8fab4bdfb62 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:16.247725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a107a8-e79c-4fa9-97ae-b8fab4bdfb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:16.270740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-66a107a8-e79c-4fa9-97ae-b8fab4bdfb62 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:16.272039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-66a107a8-e79c-4fa9-97ae-b8fab4bdfb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:16.353513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 600/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:16.378952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce7a90-b858-4451-b5d1-ead6f0e8ad3a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:16.509897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce7a90-b858-4451-b5d1-ead6f0e8ad3a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:16.553420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e782f6-cd02-4cb5-9239-64cc58c619c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:16.554111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e782f6-cd02-4cb5-9239-64cc58c619c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:16.554505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e782f6-cd02-4cb5-9239-64cc58c619c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:16.554696 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e782f6-cd02-4cb5-9239-64cc58c619c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:16.566572 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e782f6-cd02-4cb5-9239-64cc58c619c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-98e782f6-cd02-4cb5-9239-64cc58c619c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:48:16.571859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:16.572931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:16.579786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 608/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:48:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:16.580564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a107a8-e79c-4fa9-97ae-b8fab4bdfb62 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:16.628427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:16.697477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a107a8-e79c-4fa9-97ae-b8fab4bdfb62 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:16.740631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcaed0-dbe4-4b60-b0b6-926062a5c589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:16.741342 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcaed0-dbe4-4b60-b0b6-926062a5c589 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:16.742050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcaed0-dbe4-4b60-b0b6-926062a5c589 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:16.742880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcaed0-dbe4-4b60-b0b6-926062a5c589 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:16.743412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcaed0-dbe4-4b60-b0b6-926062a5c589 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:16.743880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcaed0-dbe4-4b60-b0b6-926062a5c589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:16.746009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcaed0-dbe4-4b60-b0b6-926062a5c589 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:16.761649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 601/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:48:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:48:16.763412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e782f6-cd02-4cb5-9239-64cc58c619c8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:16.821394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:16.881071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e782f6-cd02-4cb5-9239-64cc58c619c8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:16.922124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd58810-4289-4be3-a13b-8368ac652b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:16.926025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd58810-4289-4be3-a13b-8368ac652b4a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:16.926315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd58810-4289-4be3-a13b-8368ac652b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:16.926619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd58810-4289-4be3-a13b-8368ac652b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:16.939989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd58810-4289-4be3-a13b-8368ac652b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:16.941520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd58810-4289-4be3-a13b-8368ac652b4a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bfd58810-4289-4be3-a13b-8368ac652b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:16.960259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:16.961714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:16.963444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:16.963444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:16.963444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:16.963444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:16.965520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 609/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:48:16.972222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcaed0-dbe4-4b60-b0b6-926062a5c589 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:17.095034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:17.098214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcaed0-dbe4-4b60-b0b6-926062a5c589 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:17.139462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a760628-a175-4e74-a50e-6989c6c36696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:17.140993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a760628-a175-4e74-a50e-6989c6c36696 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:17.141437 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a760628-a175-4e74-a50e-6989c6c36696 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:17.141860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a760628-a175-4e74-a50e-6989c6c36696 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:17.142778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a760628-a175-4e74-a50e-6989c6c36696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:17.143543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a760628-a175-4e74-a50e-6989c6c36696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:17.145036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a760628-a175-4e74-a50e-6989c6c36696 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:17.164885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 602/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:48:17.166407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd58810-4289-4be3-a13b-8368ac652b4a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:17.289743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd58810-4289-4be3-a13b-8368ac652b4a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:17.291395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:17.330083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087e36f-8b6a-4674-bd2b-55567d951182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:17.330865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087e36f-8b6a-4674-bd2b-55567d951182 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:17.331146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087e36f-8b6a-4674-bd2b-55567d951182 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:17.331423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087e36f-8b6a-4674-bd2b-55567d951182 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:17.340178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087e36f-8b6a-4674-bd2b-55567d951182 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:17.341692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087e36f-8b6a-4674-bd2b-55567d951182 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9087e36f-8b6a-4674-bd2b-55567d951182 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:48:17.355274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:17.356578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:17.357811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:17.362051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 610/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:17.363457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a760628-a175-4e74-a50e-6989c6c36696 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:17.485619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a760628-a175-4e74-a50e-6989c6c36696 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:17.528611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a88f1-633a-4858-a6e8-88ca63977a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:17.529645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a88f1-633a-4858-a6e8-88ca63977a09 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:17.530181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a88f1-633a-4858-a6e8-88ca63977a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:17.530301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a88f1-633a-4858-a6e8-88ca63977a09 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:17.531313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a88f1-633a-4858-a6e8-88ca63977a09 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:17.531641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a88f1-633a-4858-a6e8-88ca63977a09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:17.533637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a88f1-633a-4858-a6e8-88ca63977a09 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:17.556301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 603/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Jul 30 04:48:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:17.562784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087e36f-8b6a-4674-bd2b-55567d951182 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:17.684993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087e36f-8b6a-4674-bd2b-55567d951182 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:17.729398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae61ef-4bd2-4b30-965d-b61e8d840dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:17.730425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae61ef-4bd2-4b30-965d-b61e8d840dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:17.732838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae61ef-4bd2-4b30-965d-b61e8d840dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:17.732838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae61ef-4bd2-4b30-965d-b61e8d840dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:17.770587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:17.815030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae61ef-4bd2-4b30-965d-b61e8d840dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:17.816510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae61ef-4bd2-4b30-965d-b61e8d840dc3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:17.935176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 611/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:48:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 30 04:48:17.941761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a88f1-633a-4858-a6e8-88ca63977a09 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:17.964881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:18.066226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a88f1-633a-4858-a6e8-88ca63977a09 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:18.108988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22550c08-ddce-4203-aa13-dec2575bf2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:18.109746 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22550c08-ddce-4203-aa13-dec2575bf2ae None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:18.110111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22550c08-ddce-4203-aa13-dec2575bf2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:18.110476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22550c08-ddce-4203-aa13-dec2575bf2ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:18.121153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22550c08-ddce-4203-aa13-dec2575bf2ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:18.122715 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22550c08-ddce-4203-aa13-dec2575bf2ae None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-22550c08-ddce-4203-aa13-dec2575bf2ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:18.138736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:18.140756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:18.141907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:18.141907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:18.141907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:18.141907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:18.142419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 604/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:18.148292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae61ef-4bd2-4b30-965d-b61e8d840dc3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:18.215997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:18.282386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae61ef-4bd2-4b30-965d-b61e8d840dc3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:18.334414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23e87-55b5-4be2-ad76-583951489cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:18.334414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23e87-55b5-4be2-ad76-583951489cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:18.334414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23e87-55b5-4be2-ad76-583951489cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:18.334414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23e87-55b5-4be2-ad76-583951489cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:18.335275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23e87-55b5-4be2-ad76-583951489cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:18.335275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23e87-55b5-4be2-ad76-583951489cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:18.335275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23e87-55b5-4be2-ad76-583951489cb5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:18.346844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 612/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:48:18.353366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22550c08-ddce-4203-aa13-dec2575bf2ae None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:18.444138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:18.485269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22550c08-ddce-4203-aa13-dec2575bf2ae None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:18.532220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca70e33-85cf-4130-b636-9258c4e41730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:18.533185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca70e33-85cf-4130-b636-9258c4e41730 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:18.533504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca70e33-85cf-4130-b636-9258c4e41730 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:18.533979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca70e33-85cf-4130-b636-9258c4e41730 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:18.542921 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca70e33-85cf-4130-b636-9258c4e41730 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:18.544627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca70e33-85cf-4130-b636-9258c4e41730 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-aca70e33-85cf-4130-b636-9258c4e41730 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:48:18.559051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:18.560539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:18.561979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:48:18.561979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:48:18.561979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:48:18.561979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:48:18.561979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:48:18.561979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:48:18.561979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:18.561979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:18.562789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 605/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:18.569210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23e87-55b5-4be2-ad76-583951489cb5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:18.702550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23e87-55b5-4be2-ad76-583951489cb5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:18.732602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:18.752318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3862465-29ca-448a-8295-db87d0ec1c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:18.753049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3862465-29ca-448a-8295-db87d0ec1c30 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:18.753337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3862465-29ca-448a-8295-db87d0ec1c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:18.753591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3862465-29ca-448a-8295-db87d0ec1c30 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:18.754391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3862465-29ca-448a-8295-db87d0ec1c30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:18.754763 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3862465-29ca-448a-8295-db87d0ec1c30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:18.756238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3862465-29ca-448a-8295-db87d0ec1c30 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:18.774304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 613/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:48:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:48:18.781832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca70e33-85cf-4130-b636-9258c4e41730 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:18.923873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca70e33-85cf-4130-b636-9258c4e41730 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:18.960516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:18.970004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422032b-c068-4b75-9666-5fe10c358826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:18.971043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422032b-c068-4b75-9666-5fe10c358826 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:18.971950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422032b-c068-4b75-9666-5fe10c358826 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:18.972329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422032b-c068-4b75-9666-5fe10c358826 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:18.973583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422032b-c068-4b75-9666-5fe10c358826 None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=21b7c64636bd47e6b9a486b6cdaab4b6) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:18.974056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422032b-c068-4b75-9666-5fe10c358826 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:18.975831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422032b-c068-4b75-9666-5fe10c358826 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:18.998473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 606/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:48:18 2022] GET /identity/v3/role_assignments?user.id=21b7c64636bd47e6b9a486b6cdaab4b6&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:21.163245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f798-b2cd-4941-b862-527c97ad0b97 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:21.163815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f798-b2cd-4941-b862-527c97ad0b97 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:21.163815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f798-b2cd-4941-b862-527c97ad0b97 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:21.165316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 614/1220] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:21.171912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901d77b-4d45-4db9-a6d7-88642568ab01 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:21.172550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901d77b-4d45-4db9-a6d7-88642568ab01 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:21.172550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901d77b-4d45-4db9-a6d7-88642568ab01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:21.193508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4901d77b-4d45-4db9-a6d7-88642568ab01 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:21.195005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4901d77b-4d45-4db9-a6d7-88642568ab01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:21.266534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 607/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:21.290200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87f798-b2cd-4941-b862-527c97ad0b97 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:21.401279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87f798-b2cd-4941-b862-527c97ad0b97 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:21.438704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92415954-b69f-4b52-a49c-4372d8725958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:21.439390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92415954-b69f-4b52-a49c-4372d8725958 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:21.439648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92415954-b69f-4b52-a49c-4372d8725958 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:21.439940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92415954-b69f-4b52-a49c-4372d8725958 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:21.450927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92415954-b69f-4b52-a49c-4372d8725958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-92415954-b69f-4b52-a49c-4372d8725958 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:48:21.457249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:21.458481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:21.459372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 615/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:48:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:21.464863 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901d77b-4d45-4db9-a6d7-88642568ab01 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:21.571305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901d77b-4d45-4db9-a6d7-88642568ab01 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:21.598869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:21.610782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ce957-f004-424e-bd2e-c9355f239c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:21.611406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ce957-f004-424e-bd2e-c9355f239c89 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:21.611686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ce957-f004-424e-bd2e-c9355f239c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:21.611981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ce957-f004-424e-bd2e-c9355f239c89 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:21.612695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ce957-f004-424e-bd2e-c9355f239c89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:21.612979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ce957-f004-424e-bd2e-c9355f239c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:21.614345 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ce957-f004-424e-bd2e-c9355f239c89 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:21.622736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 608/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:48:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:21.629153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92415954-b69f-4b52-a49c-4372d8725958 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:21.680147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:21.751239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92415954-b69f-4b52-a49c-4372d8725958 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:21.793784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedbae9-507a-4a2b-b7b3-b0c3a94c43ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:21.794427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedbae9-507a-4a2b-b7b3-b0c3a94c43ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:21.794739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedbae9-507a-4a2b-b7b3-b0c3a94c43ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:21.795013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedbae9-507a-4a2b-b7b3-b0c3a94c43ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:21.805938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedbae9-507a-4a2b-b7b3-b0c3a94c43ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:21.807988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedbae9-507a-4a2b-b7b3-b0c3a94c43ad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eeedbae9-507a-4a2b-b7b3-b0c3a94c43ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:21.825502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:21.826687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:21.827802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:21.827802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:21.827802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:21.827802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:21.831497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 616/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:21.832764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ce957-f004-424e-bd2e-c9355f239c89 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:21.948344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ce957-f004-424e-bd2e-c9355f239c89 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:21.986221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d576b5c-963e-4025-820e-04778c1a3faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:21.987002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d576b5c-963e-4025-820e-04778c1a3faa None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:21.987276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d576b5c-963e-4025-820e-04778c1a3faa None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:21.987618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d576b5c-963e-4025-820e-04778c1a3faa None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:21.988492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d576b5c-963e-4025-820e-04778c1a3faa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:21.988782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d576b5c-963e-4025-820e-04778c1a3faa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:21.990280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d576b5c-963e-4025-820e-04778c1a3faa None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:22.003825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 609/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:22.009556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedbae9-507a-4a2b-b7b3-b0c3a94c43ad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:22.107280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:22.114672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedbae9-507a-4a2b-b7b3-b0c3a94c43ad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:22.155353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425a000-ab4c-4d39-879e-f82f3129ceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:22.155997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425a000-ab4c-4d39-879e-f82f3129ceb6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:22.156274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425a000-ab4c-4d39-879e-f82f3129ceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:22.156542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425a000-ab4c-4d39-879e-f82f3129ceb6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:22.164447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425a000-ab4c-4d39-879e-f82f3129ceb6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:22.165945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425a000-ab4c-4d39-879e-f82f3129ceb6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2425a000-ab4c-4d39-879e-f82f3129ceb6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:48:22.178243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:22.179323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:22.180382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 617/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:22.185483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d576b5c-963e-4025-820e-04778c1a3faa None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:22.235434 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:22.297667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d576b5c-963e-4025-820e-04778c1a3faa None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:22.340844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd07e3-44cd-49eb-8745-6a7229c05f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:22.341494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd07e3-44cd-49eb-8745-6a7229c05f8c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:22.341777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd07e3-44cd-49eb-8745-6a7229c05f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:22.342617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd07e3-44cd-49eb-8745-6a7229c05f8c None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:22.343484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd07e3-44cd-49eb-8745-6a7229c05f8c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:22.343821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd07e3-44cd-49eb-8745-6a7229c05f8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:22.345322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd07e3-44cd-49eb-8745-6a7229c05f8c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:22.364759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 610/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Jul 30 04:48:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:22.370478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425a000-ab4c-4d39-879e-f82f3129ceb6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:22.483658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425a000-ab4c-4d39-879e-f82f3129ceb6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:22.522600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6adc9-9dd6-4572-a160-dec3ad566982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:22.523275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6adc9-9dd6-4572-a160-dec3ad566982 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:22.523521 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6adc9-9dd6-4572-a160-dec3ad566982 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:22.523842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6adc9-9dd6-4572-a160-dec3ad566982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:22.535260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6adc9-9dd6-4572-a160-dec3ad566982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:22.536728 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6adc9-9dd6-4572-a160-dec3ad566982 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5db6adc9-9dd6-4572-a160-dec3ad566982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:22.553109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:22.554320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:22.560096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:22.560096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:22.560096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:22.560096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:22.560096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 618/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:22.560442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd07e3-44cd-49eb-8745-6a7229c05f8c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:22.677445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd07e3-44cd-49eb-8745-6a7229c05f8c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:22.679733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:22.723496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4265087-2179-4372-9e8a-d46318460dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:22.724085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4265087-2179-4372-9e8a-d46318460dfe None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:22.724367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4265087-2179-4372-9e8a-d46318460dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:22.724672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4265087-2179-4372-9e8a-d46318460dfe None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:22.725383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4265087-2179-4372-9e8a-d46318460dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:22.725721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4265087-2179-4372-9e8a-d46318460dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:22.727253 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4265087-2179-4372-9e8a-d46318460dfe None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:22.741159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 611/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:48:22.748012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6adc9-9dd6-4572-a160-dec3ad566982 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:22.758552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:22.868288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6adc9-9dd6-4572-a160-dec3ad566982 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:22.907347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4403a2-f5ca-4168-8540-7a727b055307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:22.908030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4403a2-f5ca-4168-8540-7a727b055307 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:22.908307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4403a2-f5ca-4168-8540-7a727b055307 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:22.908619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4403a2-f5ca-4168-8540-7a727b055307 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:22.915383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4403a2-f5ca-4168-8540-7a727b055307 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:22.916814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4403a2-f5ca-4168-8540-7a727b055307 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ef4403a2-f5ca-4168-8540-7a727b055307 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:48:22.929016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:22.930167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:22.936343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:48:22.936343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:48:22.936343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:48:22.936343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:48:22.936343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:48:22.936343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:48:22.936343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:22.936343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:22.936343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 619/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:48:22.937269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4265087-2179-4372-9e8a-d46318460dfe None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:23.050250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4265087-2179-4372-9e8a-d46318460dfe None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:23.087947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba468b-745f-4c94-87eb-9f3021560a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:23.088586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba468b-745f-4c94-87eb-9f3021560a0b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:23.088847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba468b-745f-4c94-87eb-9f3021560a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:23.089121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba468b-745f-4c94-87eb-9f3021560a0b None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:23.089867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba468b-745f-4c94-87eb-9f3021560a0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:23.090169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba468b-745f-4c94-87eb-9f3021560a0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:23.091655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba468b-745f-4c94-87eb-9f3021560a0b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:23.109506 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 612/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:48:23.110808 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4403a2-f5ca-4168-8540-7a727b055307 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:23.177763 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:23.217087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4403a2-f5ca-4168-8540-7a727b055307 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:23.256407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfb0ad-6b63-48be-88ee-663aca99d911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:23.257114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfb0ad-6b63-48be-88ee-663aca99d911 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:23.257399 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfb0ad-6b63-48be-88ee-663aca99d911 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:23.257677 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfb0ad-6b63-48be-88ee-663aca99d911 None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/21b7c64636bd47e6b9a486b6cdaab4b6/roles/4785312981e946918a58301ed5112c34` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:23.277241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfb0ad-6b63-48be-88ee-663aca99d911 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=21b7c64636bd47e6b9a486b6cdaab4b6, role_id=4785312981e946918a58301ed5112c34)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:23.278847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfb0ad-6b63-48be-88ee-663aca99d911 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:23.295633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 620/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:48:23 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/21b7c64636bd47e6b9a486b6cdaab4b6/roles/4785312981e946918a58301ed5112c34 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:48:25.486906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe463669-3e78-4519-bad0-b74e176d5652 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:25.487457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe463669-3e78-4519-bad0-b74e176d5652 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:25.487457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe463669-3e78-4519-bad0-b74e176d5652 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:25.488937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 613/1233] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:25.494609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774e5bd-bd08-4f2a-8ae3-a423ce328a0a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:25.495046 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774e5bd-bd08-4f2a-8ae3-a423ce328a0a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:25.495196 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774e5bd-bd08-4f2a-8ae3-a423ce328a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:25.514972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5774e5bd-bd08-4f2a-8ae3-a423ce328a0a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:25.516163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5774e5bd-bd08-4f2a-8ae3-a423ce328a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:25.587539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 621/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:25.611706 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe463669-3e78-4519-bad0-b74e176d5652 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:25.637892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:25.719019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe463669-3e78-4519-bad0-b74e176d5652 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:25.756687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a78a2-7731-4cc7-8995-eeba270c7e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:25.757193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a78a2-7731-4cc7-8995-eeba270c7e83 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:25.757496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a78a2-7731-4cc7-8995-eeba270c7e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:25.757752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a78a2-7731-4cc7-8995-eeba270c7e83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:25.772654 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a78a2-7731-4cc7-8995-eeba270c7e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ce0a78a2-7731-4cc7-8995-eeba270c7e83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:48:25.777946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:25.780122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:25.781326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 614/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Jul 30 04:48:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:25.786115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774e5bd-bd08-4f2a-8ae3-a423ce328a0a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:25.839005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:25.897806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774e5bd-bd08-4f2a-8ae3-a423ce328a0a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:25.936969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59657aff-6d6f-4bf4-9d29-b9910b5c3cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:25.937601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59657aff-6d6f-4bf4-9d29-b9910b5c3cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:25.937887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59657aff-6d6f-4bf4-9d29-b9910b5c3cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:25.938163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59657aff-6d6f-4bf4-9d29-b9910b5c3cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:25.938914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59657aff-6d6f-4bf4-9d29-b9910b5c3cb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:25.939214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59657aff-6d6f-4bf4-9d29-b9910b5c3cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:25.940581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59657aff-6d6f-4bf4-9d29-b9910b5c3cb0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:25.950485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 622/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Jul 30 04:48:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:25.956844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a78a2-7731-4cc7-8995-eeba270c7e83 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:26.052792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:26.070428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a78a2-7731-4cc7-8995-eeba270c7e83 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:26.107930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900230b0-5d93-4652-b755-a5ac69a2711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:26.108639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900230b0-5d93-4652-b755-a5ac69a2711d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:26.108951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900230b0-5d93-4652-b755-a5ac69a2711d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:26.109283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900230b0-5d93-4652-b755-a5ac69a2711d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:26.121134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900230b0-5d93-4652-b755-a5ac69a2711d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:26.122960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900230b0-5d93-4652-b755-a5ac69a2711d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-900230b0-5d93-4652-b755-a5ac69a2711d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:26.139346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:26.140538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:26.141467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:26.141467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:26.141467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:26.141467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:26.141467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 615/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:26.146849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59657aff-6d6f-4bf4-9d29-b9910b5c3cb0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:26.261223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59657aff-6d6f-4bf4-9d29-b9910b5c3cb0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:26.272666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:26.304492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d3960-3db8-4d0f-b89a-7c9e69cf0452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:26.305199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d3960-3db8-4d0f-b89a-7c9e69cf0452 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:26.305575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d3960-3db8-4d0f-b89a-7c9e69cf0452 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:26.305797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d3960-3db8-4d0f-b89a-7c9e69cf0452 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:26.306585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d3960-3db8-4d0f-b89a-7c9e69cf0452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:26.306937 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d3960-3db8-4d0f-b89a-7c9e69cf0452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:26.308494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d3960-3db8-4d0f-b89a-7c9e69cf0452 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:26.321638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 623/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:26.327922 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900230b0-5d93-4652-b755-a5ac69a2711d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:26.448421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900230b0-5d93-4652-b755-a5ac69a2711d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:26.490596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d906e85-18f7-4e74-baf4-874ce3b2d604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:26.491296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d906e85-18f7-4e74-baf4-874ce3b2d604 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:26.491513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d906e85-18f7-4e74-baf4-874ce3b2d604 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:26.491805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d906e85-18f7-4e74-baf4-874ce3b2d604 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:26.495403 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:26.502588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d906e85-18f7-4e74-baf4-874ce3b2d604 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:26.504503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d906e85-18f7-4e74-baf4-874ce3b2d604 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3d906e85-18f7-4e74-baf4-874ce3b2d604 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:48:26.518558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:26.519774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:26.520787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 616/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:26.526051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d3960-3db8-4d0f-b89a-7c9e69cf0452 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:26.639421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d3960-3db8-4d0f-b89a-7c9e69cf0452 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:26.678386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ade7f-3316-4ae3-95ae-3803bb04ba71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:26.679179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ade7f-3316-4ae3-95ae-3803bb04ba71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:26.679488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ade7f-3316-4ae3-95ae-3803bb04ba71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:26.679837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ade7f-3316-4ae3-95ae-3803bb04ba71 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:26.680885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ade7f-3316-4ae3-95ae-3803bb04ba71 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:26.681369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ade7f-3316-4ae3-95ae-3803bb04ba71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:26.682970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ade7f-3316-4ae3-95ae-3803bb04ba71 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:26.708240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 624/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Jul 30 04:48:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:26.713381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d906e85-18f7-4e74-baf4-874ce3b2d604 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:26.829361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d906e85-18f7-4e74-baf4-874ce3b2d604 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:26.865810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1efdfa-11d9-41c4-903d-105c29b2f635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:26.866429 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1efdfa-11d9-41c4-903d-105c29b2f635 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:26.866807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1efdfa-11d9-41c4-903d-105c29b2f635 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:26.867006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1efdfa-11d9-41c4-903d-105c29b2f635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:26.934902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:26.942946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1efdfa-11d9-41c4-903d-105c29b2f635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:26.944656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1efdfa-11d9-41c4-903d-105c29b2f635 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:27.058907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 617/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:48:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 30 04:48:27.065022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ade7f-3316-4ae3-95ae-3803bb04ba71 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:27.097253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:27.184661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ade7f-3316-4ae3-95ae-3803bb04ba71 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:27.224270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41815f-31f6-4f78-a28f-59fcf4769cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:27.225040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41815f-31f6-4f78-a28f-59fcf4769cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:27.225325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41815f-31f6-4f78-a28f-59fcf4769cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:27.225598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41815f-31f6-4f78-a28f-59fcf4769cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:27.238061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41815f-31f6-4f78-a28f-59fcf4769cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:27.239686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41815f-31f6-4f78-a28f-59fcf4769cf3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aa41815f-31f6-4f78-a28f-59fcf4769cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:27.258085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:27.259125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:27.260060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:27.260060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:27.260060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:27.260060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:27.260060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 625/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:48:27.266428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1efdfa-11d9-41c4-903d-105c29b2f635 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:27.375563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:27.381842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1efdfa-11d9-41c4-903d-105c29b2f635 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:27.419542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c3221-58d7-4a56-b67b-50ce703271d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:27.420236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c3221-58d7-4a56-b67b-50ce703271d1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:27.420522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c3221-58d7-4a56-b67b-50ce703271d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:27.420796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c3221-58d7-4a56-b67b-50ce703271d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:27.421561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c3221-58d7-4a56-b67b-50ce703271d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:27.421890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c3221-58d7-4a56-b67b-50ce703271d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:27.423406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c3221-58d7-4a56-b67b-50ce703271d1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:27.437355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 618/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:27.443394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41815f-31f6-4f78-a28f-59fcf4769cf3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:27.535080 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:27.561101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41815f-31f6-4f78-a28f-59fcf4769cf3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:27.595831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb7e19-0d36-4f5e-904f-70dfb69913b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:27.596462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb7e19-0d36-4f5e-904f-70dfb69913b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:27.596739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb7e19-0d36-4f5e-904f-70dfb69913b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:27.597007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb7e19-0d36-4f5e-904f-70dfb69913b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:27.605579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb7e19-0d36-4f5e-904f-70dfb69913b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:27.607798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb7e19-0d36-4f5e-904f-70dfb69913b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8edb7e19-0d36-4f5e-904f-70dfb69913b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:48:27.620125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:27.621542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:27.623455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:48:27.623455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:48:27.623455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:48:27.623455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:48:27.623455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:48:27.623455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:48:27.623455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:27.623455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:27.623455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 626/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:48:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:27.628589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c3221-58d7-4a56-b67b-50ce703271d1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:27.744736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c3221-58d7-4a56-b67b-50ce703271d1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:27.784131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65b7f6-83c0-4973-aefd-71032fa63981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:27.784808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b7f6-83c0-4973-aefd-71032fa63981 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:27.785085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b7f6-83c0-4973-aefd-71032fa63981 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:27.785353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b7f6-83c0-4973-aefd-71032fa63981 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:27.786300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65b7f6-83c0-4973-aefd-71032fa63981 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:27.786658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65b7f6-83c0-4973-aefd-71032fa63981 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:27.788367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65b7f6-83c0-4973-aefd-71032fa63981 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:27.801898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 619/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:48:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:27.808903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb7e19-0d36-4f5e-904f-70dfb69913b6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:27.923589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb7e19-0d36-4f5e-904f-70dfb69913b6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:27.962370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9d5b-9852-4666-958e-8dc413a6abef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:27.963023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9d5b-9852-4666-958e-8dc413a6abef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:27.963304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9d5b-9852-4666-958e-8dc413a6abef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:27.963580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9d5b-9852-4666-958e-8dc413a6abef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:27.964364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9d5b-9852-4666-958e-8dc413a6abef None admin] RBAC: Adding query filter params (role.id=4785312981e946918a58301ed5112c34, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=21b7c64636bd47e6b9a486b6cdaab4b6) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:27.964672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9d5b-9852-4666-958e-8dc413a6abef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:27.966255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9d5b-9852-4666-958e-8dc413a6abef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:27.972981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 627/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:48:27 2022] GET /identity/v3/role_assignments?user.id=21b7c64636bd47e6b9a486b6cdaab4b6&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=4785312981e946918a58301ed5112c34 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:30.164334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344a11c-2eb2-4ca0-bcaf-f4bccb5e3e0a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:30.164860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344a11c-2eb2-4ca0-bcaf-f4bccb5e3e0a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:30.164860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344a11c-2eb2-4ca0-bcaf-f4bccb5e3e0a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:30.166755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 620/1247] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:30.173279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46465235-c000-4381-a224-4ca3acaf5ca7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:30.173798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46465235-c000-4381-a224-4ca3acaf5ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:30.173971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46465235-c000-4381-a224-4ca3acaf5ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:30.181058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:30.195929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-46465235-c000-4381-a224-4ca3acaf5ca7 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:30.197150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-46465235-c000-4381-a224-4ca3acaf5ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:30.271466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 628/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:30.295170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344a11c-2eb2-4ca0-bcaf-f4bccb5e3e0a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:30.315858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:30.420968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344a11c-2eb2-4ca0-bcaf-f4bccb5e3e0a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:30.470286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99919b55-a6d8-4170-b9a2-ea45220c4ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:30.470995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99919b55-a6d8-4170-b9a2-ea45220c4ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:30.471258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99919b55-a6d8-4170-b9a2-ea45220c4ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:30.471539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99919b55-a6d8-4170-b9a2-ea45220c4ef9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:30.482175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99919b55-a6d8-4170-b9a2-ea45220c4ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-99919b55-a6d8-4170-b9a2-ea45220c4ef9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:48:30.488172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:30.489308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:30.490182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 621/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:48:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:30.495786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46465235-c000-4381-a224-4ca3acaf5ca7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:30.611570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46465235-c000-4381-a224-4ca3acaf5ca7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:30.650123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:30.658041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd870f-af3c-4c77-aaee-51efbb817926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:30.658704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd870f-af3c-4c77-aaee-51efbb817926 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:30.658995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd870f-af3c-4c77-aaee-51efbb817926 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:30.659269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd870f-af3c-4c77-aaee-51efbb817926 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:30.660118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd870f-af3c-4c77-aaee-51efbb817926 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:30.660397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd870f-af3c-4c77-aaee-51efbb817926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:30.661863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd870f-af3c-4c77-aaee-51efbb817926 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:30.671677 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 629/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:48:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:48:30.677344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99919b55-a6d8-4170-b9a2-ea45220c4ef9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:30.769430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:30.793809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99919b55-a6d8-4170-b9a2-ea45220c4ef9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:30.832739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67549-87c6-4c9e-93b9-5bcfd1e3f729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:30.833390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67549-87c6-4c9e-93b9-5bcfd1e3f729 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:30.833667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67549-87c6-4c9e-93b9-5bcfd1e3f729 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:30.833941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67549-87c6-4c9e-93b9-5bcfd1e3f729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:30.844816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db67549-87c6-4c9e-93b9-5bcfd1e3f729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:30.846460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db67549-87c6-4c9e-93b9-5bcfd1e3f729 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8db67549-87c6-4c9e-93b9-5bcfd1e3f729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:30.864852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:30.865984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:30.867117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:30.867117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:30.867117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:30.867117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:30.867117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 622/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 30 04:48:30.873389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd870f-af3c-4c77-aaee-51efbb817926 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:30.989907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd870f-af3c-4c77-aaee-51efbb817926 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:31.030577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f94bc-b067-47c7-9ae3-8f011033a178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:31.031245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f94bc-b067-47c7-9ae3-8f011033a178 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:31.031775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f94bc-b067-47c7-9ae3-8f011033a178 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:31.031891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f94bc-b067-47c7-9ae3-8f011033a178 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:31.032755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f94bc-b067-47c7-9ae3-8f011033a178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:31.033188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f94bc-b067-47c7-9ae3-8f011033a178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:31.034697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f94bc-b067-47c7-9ae3-8f011033a178 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:31.049292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 630/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:31.054837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67549-87c6-4c9e-93b9-5bcfd1e3f729 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:31.171813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67549-87c6-4c9e-93b9-5bcfd1e3f729 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:31.201673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:31.210168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d50f19-ec11-4888-b700-cce4595fa7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:31.210872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d50f19-ec11-4888-b700-cce4595fa7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:31.211158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d50f19-ec11-4888-b700-cce4595fa7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:31.211436 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d50f19-ec11-4888-b700-cce4595fa7f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:31.220163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d50f19-ec11-4888-b700-cce4595fa7f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:31.221774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d50f19-ec11-4888-b700-cce4595fa7f1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-33d50f19-ec11-4888-b700-cce4595fa7f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:48:31.234174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:31.240108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:31.241447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 623/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:31.242185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f94bc-b067-47c7-9ae3-8f011033a178 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:31.308109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:31.360034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f94bc-b067-47c7-9ae3-8f011033a178 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:31.399398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20e060-109f-4ff5-a990-cd32bbbe6ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:31.400017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20e060-109f-4ff5-a990-cd32bbbe6ccc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:31.400299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20e060-109f-4ff5-a990-cd32bbbe6ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:31.400560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20e060-109f-4ff5-a990-cd32bbbe6ccc None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:31.401349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20e060-109f-4ff5-a990-cd32bbbe6ccc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:31.401657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20e060-109f-4ff5-a990-cd32bbbe6ccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:31.403112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20e060-109f-4ff5-a990-cd32bbbe6ccc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:31.421960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 631/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Jul 30 04:48:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:48:31.428602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d50f19-ec11-4888-b700-cce4595fa7f1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:31.541096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d50f19-ec11-4888-b700-cce4595fa7f1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:31.578883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdaaa7-b45c-4220-a657-6827da0f07d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:31.579496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdaaa7-b45c-4220-a657-6827da0f07d8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:31.579773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdaaa7-b45c-4220-a657-6827da0f07d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:31.580044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdaaa7-b45c-4220-a657-6827da0f07d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:31.646680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:31.663544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdaaa7-b45c-4220-a657-6827da0f07d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:31.665010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdaaa7-b45c-4220-a657-6827da0f07d8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:31.781729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 624/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:48:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:48:31.787520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20e060-109f-4ff5-a990-cd32bbbe6ccc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:31.904464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20e060-109f-4ff5-a990-cd32bbbe6ccc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:31.922886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:31.948143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd57b23-dc2f-4e9c-9961-072167f77ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:31.948789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd57b23-dc2f-4e9c-9961-072167f77ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:31.949086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd57b23-dc2f-4e9c-9961-072167f77ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:31.949342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd57b23-dc2f-4e9c-9961-072167f77ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:31.960514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd57b23-dc2f-4e9c-9961-072167f77ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:31.962076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd57b23-dc2f-4e9c-9961-072167f77ca0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ddd57b23-dc2f-4e9c-9961-072167f77ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:31.977742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:31.978936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:31.979836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:31.979836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:31.979836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:31.979836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:31.979836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 632/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:31.984879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdaaa7-b45c-4220-a657-6827da0f07d8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:32.082470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:32.096086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdaaa7-b45c-4220-a657-6827da0f07d8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:32.137414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1127a-ae9e-4b81-a720-69e420b5cb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:32.138244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1127a-ae9e-4b81-a720-69e420b5cb39 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:32.138560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1127a-ae9e-4b81-a720-69e420b5cb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:32.138840 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1127a-ae9e-4b81-a720-69e420b5cb39 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:32.139751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1127a-ae9e-4b81-a720-69e420b5cb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:32.140100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1127a-ae9e-4b81-a720-69e420b5cb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:32.142293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1127a-ae9e-4b81-a720-69e420b5cb39 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:32.154203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 625/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:32.160459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd57b23-dc2f-4e9c-9961-072167f77ca0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:32.283530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd57b23-dc2f-4e9c-9961-072167f77ca0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:32.319423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3909-d84e-4404-b56f-00d655304682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:32.320102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3909-d84e-4404-b56f-00d655304682 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:32.320388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3909-d84e-4404-b56f-00d655304682 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:32.320701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3909-d84e-4404-b56f-00d655304682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:32.330601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e3909-d84e-4404-b56f-00d655304682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:32.332045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e3909-d84e-4404-b56f-00d655304682 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c25e3909-d84e-4404-b56f-00d655304682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:48:32.346749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:32.347895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:32.349221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:48:32.349221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:48:32.349221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:48:32.349221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:48:32.349221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:48:32.349221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:48:32.349221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:32.349221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:32.349221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 633/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:48:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:32.354044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1127a-ae9e-4b81-a720-69e420b5cb39 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:32.468420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1127a-ae9e-4b81-a720-69e420b5cb39 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:32.505065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d818c46-534f-4a3c-a86a-d353f9df39d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:32.505683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d818c46-534f-4a3c-a86a-d353f9df39d0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:32.505966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d818c46-534f-4a3c-a86a-d353f9df39d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:32.506270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d818c46-534f-4a3c-a86a-d353f9df39d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:32.507015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d818c46-534f-4a3c-a86a-d353f9df39d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:32.507309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d818c46-534f-4a3c-a86a-d353f9df39d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:32.508770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d818c46-534f-4a3c-a86a-d353f9df39d0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:32.524594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 626/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:48:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:32.531457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3909-d84e-4404-b56f-00d655304682 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:32.558862 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:32.647630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3909-d84e-4404-b56f-00d655304682 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:32.684444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7636a-2b8b-417e-9969-39968333f997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:32.685171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7636a-2b8b-417e-9969-39968333f997 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:32.685480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7636a-2b8b-417e-9969-39968333f997 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:32.685754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7636a-2b8b-417e-9969-39968333f997 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:32.686774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7636a-2b8b-417e-9969-39968333f997 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=21b7c64636bd47e6b9a486b6cdaab4b6) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:32.687109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7636a-2b8b-417e-9969-39968333f997 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:32.688735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7636a-2b8b-417e-9969-39968333f997 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:32.695233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 634/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:48:32 2022] GET /identity/v3/role_assignments?user.id=21b7c64636bd47e6b9a486b6cdaab4b6&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:34.669138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef31161-f5fc-43fb-8d2c-7f9051e0b295 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:34.669712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef31161-f5fc-43fb-8d2c-7f9051e0b295 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:34.669712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef31161-f5fc-43fb-8d2c-7f9051e0b295 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:34.671118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 627/1261] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:34.676193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa8be-23e0-4818-b3ba-6b6a2c6e7a75 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:34.676636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa8be-23e0-4818-b3ba-6b6a2c6e7a75 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:34.676744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa8be-23e0-4818-b3ba-6b6a2c6e7a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:34.695048 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c38fa8be-23e0-4818-b3ba-6b6a2c6e7a75 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:34.696232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c38fa8be-23e0-4818-b3ba-6b6a2c6e7a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:34.777867 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 635/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:34.799528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef31161-f5fc-43fb-8d2c-7f9051e0b295 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:34.810722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:34.898420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef31161-f5fc-43fb-8d2c-7f9051e0b295 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:34.930620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9d106-ebff-4db6-b76e-292bfdee2a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:34.931185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9d106-ebff-4db6-b76e-292bfdee2a4d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:34.931462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9d106-ebff-4db6-b76e-292bfdee2a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:34.931729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9d106-ebff-4db6-b76e-292bfdee2a4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:34.941948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9d106-ebff-4db6-b76e-292bfdee2a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e2f9d106-ebff-4db6-b76e-292bfdee2a4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:48:34.946792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:34.947933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:34.948848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 628/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:34.954501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fa8be-23e0-4818-b3ba-6b6a2c6e7a75 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:34.972356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:35.052123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fa8be-23e0-4818-b3ba-6b6a2c6e7a75 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:35.085066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e53df-1457-4f87-b180-76e133876108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:35.085666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e53df-1457-4f87-b180-76e133876108 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:35.085940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e53df-1457-4f87-b180-76e133876108 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:35.086207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e53df-1457-4f87-b180-76e133876108 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:35.086942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e53df-1457-4f87-b180-76e133876108 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:35.087241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e53df-1457-4f87-b180-76e133876108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:35.088598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e53df-1457-4f87-b180-76e133876108 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:35.096364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 636/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:35.101983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9d106-ebff-4db6-b76e-292bfdee2a4d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:35.169962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:35.203423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9d106-ebff-4db6-b76e-292bfdee2a4d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:35.236323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378fb3e-ffac-4386-8e8c-74eab46963b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:35.236941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378fb3e-ffac-4386-8e8c-74eab46963b2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:35.237232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378fb3e-ffac-4386-8e8c-74eab46963b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:35.237482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378fb3e-ffac-4386-8e8c-74eab46963b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:35.248609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378fb3e-ffac-4386-8e8c-74eab46963b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:35.250049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378fb3e-ffac-4386-8e8c-74eab46963b2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8378fb3e-ffac-4386-8e8c-74eab46963b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:35.264232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:35.265276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:35.266237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:35.266237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:35.266237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:35.266237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:35.266237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 629/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:35.271301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e53df-1457-4f87-b180-76e133876108 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:35.351073 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:35.370816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e53df-1457-4f87-b180-76e133876108 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:35.404955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309a8cf-9f0e-4b19-a632-297783dc7b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:35.405560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309a8cf-9f0e-4b19-a632-297783dc7b42 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:35.405837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309a8cf-9f0e-4b19-a632-297783dc7b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:35.406109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309a8cf-9f0e-4b19-a632-297783dc7b42 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:35.406816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309a8cf-9f0e-4b19-a632-297783dc7b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:35.407146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309a8cf-9f0e-4b19-a632-297783dc7b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:35.408497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309a8cf-9f0e-4b19-a632-297783dc7b42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:35.419825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 637/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:35.425403 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378fb3e-ffac-4386-8e8c-74eab46963b2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:35.523601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378fb3e-ffac-4386-8e8c-74eab46963b2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:35.542356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:35.556315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef393afd-c424-4764-964f-ff3843469016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:35.556944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef393afd-c424-4764-964f-ff3843469016 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:35.557219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef393afd-c424-4764-964f-ff3843469016 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:35.557487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef393afd-c424-4764-964f-ff3843469016 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:35.565391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef393afd-c424-4764-964f-ff3843469016 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:35.566852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef393afd-c424-4764-964f-ff3843469016 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ef393afd-c424-4764-964f-ff3843469016 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:48:35.578360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:35.579509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:35.580631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 630/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:35.585099 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309a8cf-9f0e-4b19-a632-297783dc7b42 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:35.681351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309a8cf-9f0e-4b19-a632-297783dc7b42 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:35.718186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac8eba-4819-44f3-9b23-c859ac94de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:35.718922 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac8eba-4819-44f3-9b23-c859ac94de0a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:35.719239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac8eba-4819-44f3-9b23-c859ac94de0a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:35.719550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac8eba-4819-44f3-9b23-c859ac94de0a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:35.720389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac8eba-4819-44f3-9b23-c859ac94de0a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:35.720835 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac8eba-4819-44f3-9b23-c859ac94de0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:35.722286 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac8eba-4819-44f3-9b23-c859ac94de0a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:35.732347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:35.742892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 638/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Jul 30 04:48:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:35.749420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef393afd-c424-4764-964f-ff3843469016 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:35.849022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef393afd-c424-4764-964f-ff3843469016 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:35.882029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e42e7a-134b-4987-8c96-0f73a6eaef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:35.882682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e42e7a-134b-4987-8c96-0f73a6eaef75 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:35.882958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e42e7a-134b-4987-8c96-0f73a6eaef75 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:35.883225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e42e7a-134b-4987-8c96-0f73a6eaef75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:35.893827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e42e7a-134b-4987-8c96-0f73a6eaef75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:35.895269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e42e7a-134b-4987-8c96-0f73a6eaef75 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-78e42e7a-134b-4987-8c96-0f73a6eaef75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:35.908894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:35.909936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:35.911210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:35.911210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:35.911210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:35.911210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:35.911210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 631/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:35.915735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac8eba-4819-44f3-9b23-c859ac94de0a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:36.015279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac8eba-4819-44f3-9b23-c859ac94de0a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:36.050082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c145992-6579-4e04-9c21-33b30462967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:36.050723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c145992-6579-4e04-9c21-33b30462967f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:36.051003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c145992-6579-4e04-9c21-33b30462967f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:36.051278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c145992-6579-4e04-9c21-33b30462967f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:36.051960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c145992-6579-4e04-9c21-33b30462967f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:36.052260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c145992-6579-4e04-9c21-33b30462967f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:36.053625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c145992-6579-4e04-9c21-33b30462967f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:36.065316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 639/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:36.070904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e42e7a-134b-4987-8c96-0f73a6eaef75 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:36.101669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:36.169595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e42e7a-134b-4987-8c96-0f73a6eaef75 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:36.201453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e3a31-101f-4c8e-898f-7016079efec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:36.202069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e3a31-101f-4c8e-898f-7016079efec9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:36.202343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e3a31-101f-4c8e-898f-7016079efec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:36.202644 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e3a31-101f-4c8e-898f-7016079efec9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:36.208975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e3a31-101f-4c8e-898f-7016079efec9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:36.210400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e3a31-101f-4c8e-898f-7016079efec9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c51e3a31-101f-4c8e-898f-7016079efec9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:48:36.221907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:36.223017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:36.223966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:48:36.223966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:48:36.223966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:48:36.223966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:48:36.223966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:48:36.223966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:48:36.223966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:36.223966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:36.223966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 632/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:48:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:36.228719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c145992-6579-4e04-9c21-33b30462967f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:36.286419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:36.328744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c145992-6579-4e04-9c21-33b30462967f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:36.361436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178e093-bcc8-472f-a7f0-071285442e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:36.361985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178e093-bcc8-472f-a7f0-071285442e14 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:36.362264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178e093-bcc8-472f-a7f0-071285442e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:36.362564 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178e093-bcc8-472f-a7f0-071285442e14 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:36.363299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178e093-bcc8-472f-a7f0-071285442e14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:36.363601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178e093-bcc8-472f-a7f0-071285442e14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:36.365021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178e093-bcc8-472f-a7f0-071285442e14 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:36.377273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 640/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:48:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:36.383233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e3a31-101f-4c8e-898f-7016079efec9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:36.473203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:36.482894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e3a31-101f-4c8e-898f-7016079efec9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:36.515977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdc0d5-cb77-4ae1-887a-3419692145a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:36.516893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdc0d5-cb77-4ae1-887a-3419692145a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:36.517174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdc0d5-cb77-4ae1-887a-3419692145a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:36.517449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdc0d5-cb77-4ae1-887a-3419692145a2 None admin] PATH_INFO: `/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/21b7c64636bd47e6b9a486b6cdaab4b6/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:36.519403 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdc0d5-cb77-4ae1-887a-3419692145a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bfc8c66b48b4981832f2c4b940df124, user_id=21b7c64636bd47e6b9a486b6cdaab4b6, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:36.520836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdc0d5-cb77-4ae1-887a-3419692145a2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:36.533284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 633/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Jul 30 04:48:36 2022] PUT /identity/v3/projects/2bfc8c66b48b4981832f2c4b940df124/users/21b7c64636bd47e6b9a486b6cdaab4b6/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:48:38.536415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c08e1-c3fe-40f9-8684-1e167e9dde7f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:38.536969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c08e1-c3fe-40f9-8684-1e167e9dde7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:38.536969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c08e1-c3fe-40f9-8684-1e167e9dde7f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:38.538282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 641/1274] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:38.543838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fe5d7-bc8f-476c-8b8d-131ba02be579 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:38.544237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fe5d7-bc8f-476c-8b8d-131ba02be579 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:38.544397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fe5d7-bc8f-476c-8b8d-131ba02be579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:38.562995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-072fe5d7-bc8f-476c-8b8d-131ba02be579 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:38.564173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-072fe5d7-bc8f-476c-8b8d-131ba02be579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:38.629375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 634/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:38.653512 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c08e1-c3fe-40f9-8684-1e167e9dde7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:38.761647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c08e1-c3fe-40f9-8684-1e167e9dde7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:38.780127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:38.797445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4fd3d-ee29-4c77-905f-14336e6a7b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:38.798024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4fd3d-ee29-4c77-905f-14336e6a7b71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:38.798304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4fd3d-ee29-4c77-905f-14336e6a7b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:38.798607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4fd3d-ee29-4c77-905f-14336e6a7b71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:38.808320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4fd3d-ee29-4c77-905f-14336e6a7b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-75f4fd3d-ee29-4c77-905f-14336e6a7b71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 30 04:48:38.814165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:38.815323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:38.816402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 642/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Jul 30 04:48:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:38.821396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fe5d7-bc8f-476c-8b8d-131ba02be579 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:38.900025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:38.919899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072fe5d7-bc8f-476c-8b8d-131ba02be579 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:38.953584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b27e6-ca8f-459e-8dcd-1c2735bc463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:38.954185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b27e6-ca8f-459e-8dcd-1c2735bc463f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:38.954485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b27e6-ca8f-459e-8dcd-1c2735bc463f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:38.954759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b27e6-ca8f-459e-8dcd-1c2735bc463f None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:38.955467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b27e6-ca8f-459e-8dcd-1c2735bc463f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:38.955766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b27e6-ca8f-459e-8dcd-1c2735bc463f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:38.957120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b27e6-ca8f-459e-8dcd-1c2735bc463f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:38.965051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 635/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Jul 30 04:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:38.970712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4fd3d-ee29-4c77-905f-14336e6a7b71 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:39.072112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4fd3d-ee29-4c77-905f-14336e6a7b71 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:39.104749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689679c-0e4f-4673-9f8e-51aae72bb1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:39.105373 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689679c-0e4f-4673-9f8e-51aae72bb1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:39.106184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689679c-0e4f-4673-9f8e-51aae72bb1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:39.106471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689679c-0e4f-4673-9f8e-51aae72bb1a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:39.116312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689679c-0e4f-4673-9f8e-51aae72bb1a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:39.117730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689679c-0e4f-4673-9f8e-51aae72bb1a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7689679c-0e4f-4673-9f8e-51aae72bb1a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:39.132602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:39.133672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:39.134770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:39.134770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:39.134770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:39.134770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:39.134770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 643/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:39.140211 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b27e6-ca8f-459e-8dcd-1c2735bc463f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:39.237661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b27e6-ca8f-459e-8dcd-1c2735bc463f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:39.272094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329a60f-d333-4002-9fe5-6b420733bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:39.272705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329a60f-d333-4002-9fe5-6b420733bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:39.272984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329a60f-d333-4002-9fe5-6b420733bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:39.273257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329a60f-d333-4002-9fe5-6b420733bdd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:39.273938 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329a60f-d333-4002-9fe5-6b420733bdd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:39.274242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329a60f-d333-4002-9fe5-6b420733bdd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:39.275638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329a60f-d333-4002-9fe5-6b420733bdd3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:39.287612 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 636/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:39.293180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689679c-0e4f-4673-9f8e-51aae72bb1a3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:39.323426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:39.394938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689679c-0e4f-4673-9f8e-51aae72bb1a3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:39.427570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55640476-d207-4415-ad96-73a4e9e252a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:39.428206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55640476-d207-4415-ad96-73a4e9e252a4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:39.428489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55640476-d207-4415-ad96-73a4e9e252a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:39.428768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55640476-d207-4415-ad96-73a4e9e252a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:39.437432 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55640476-d207-4415-ad96-73a4e9e252a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:39.438908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55640476-d207-4415-ad96-73a4e9e252a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-55640476-d207-4415-ad96-73a4e9e252a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 30 04:48:39.450720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:39.451903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:39.453143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 644/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:39.457606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329a60f-d333-4002-9fe5-6b420733bdd3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:39.461350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:39.554966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329a60f-d333-4002-9fe5-6b420733bdd3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:39.587544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d354e8-00d6-4a93-87e6-133a349506a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:39.588086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d354e8-00d6-4a93-87e6-133a349506a6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:39.588359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d354e8-00d6-4a93-87e6-133a349506a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:39.588625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d354e8-00d6-4a93-87e6-133a349506a6 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:39.589394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d354e8-00d6-4a93-87e6-133a349506a6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:39.589692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d354e8-00d6-4a93-87e6-133a349506a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:39.591371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d354e8-00d6-4a93-87e6-133a349506a6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:39.608109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 637/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Jul 30 04:48:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:39.613774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55640476-d207-4415-ad96-73a4e9e252a4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:39.702265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:39.712865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55640476-d207-4415-ad96-73a4e9e252a4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:39.751590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46901e1e-f235-4a60-b4b8-7af818bfea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:39.752208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46901e1e-f235-4a60-b4b8-7af818bfea5b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:39.752488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46901e1e-f235-4a60-b4b8-7af818bfea5b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:39.752761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46901e1e-f235-4a60-b4b8-7af818bfea5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:39.819341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46901e1e-f235-4a60-b4b8-7af818bfea5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:39.820778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46901e1e-f235-4a60-b4b8-7af818bfea5b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:39.907277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:39.917652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 645/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Jul 30 04:48:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 30 04:48:39.924877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d354e8-00d6-4a93-87e6-133a349506a6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:39.986598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:40.037575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d354e8-00d6-4a93-87e6-133a349506a6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:40.071876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480850-8374-47a8-a46c-9b58c1a8ba84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:40.072503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480850-8374-47a8-a46c-9b58c1a8ba84 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:40.072778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480850-8374-47a8-a46c-9b58c1a8ba84 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:40.073047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480850-8374-47a8-a46c-9b58c1a8ba84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:40.083149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff480850-8374-47a8-a46c-9b58c1a8ba84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:40.084578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff480850-8374-47a8-a46c-9b58c1a8ba84 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ff480850-8374-47a8-a46c-9b58c1a8ba84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 30 04:48:40.099689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:40.100761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:40.101705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 30 04:48:40.101705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 30 04:48:40.101705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 30 04:48:40.101705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:40.101705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 638/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:40.107366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46901e1e-f235-4a60-b4b8-7af818bfea5b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:40.204301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46901e1e-f235-4a60-b4b8-7af818bfea5b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:40.238082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d3470-da49-461d-b8a2-3d748c9bfe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:40.238722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d3470-da49-461d-b8a2-3d748c9bfe75 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:40.239007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d3470-da49-461d-b8a2-3d748c9bfe75 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:40.239284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d3470-da49-461d-b8a2-3d748c9bfe75 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:40.239966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d3470-da49-461d-b8a2-3d748c9bfe75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:40.240295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d3470-da49-461d-b8a2-3d748c9bfe75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:40.241649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d3470-da49-461d-b8a2-3d748c9bfe75 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:40.252757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 646/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Jul 30 04:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:40.258311 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480850-8374-47a8-a46c-9b58c1a8ba84 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:40.355998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480850-8374-47a8-a46c-9b58c1a8ba84 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:40.368949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:40.389673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34c2ee-fe85-4f91-a672-d70a4f139244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:40.390295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34c2ee-fe85-4f91-a672-d70a4f139244 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:40.390601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34c2ee-fe85-4f91-a672-d70a4f139244 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:40.390873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34c2ee-fe85-4f91-a672-d70a4f139244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:40.397514 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34c2ee-fe85-4f91-a672-d70a4f139244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:40.398976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34c2ee-fe85-4f91-a672-d70a4f139244 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-be34c2ee-fe85-4f91-a672-d70a4f139244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 30 04:48:40.410533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:40.411579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:40.412705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 30 04:48:40.412705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 30 04:48:40.412705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 30 04:48:40.412705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 30 04:48:40.412705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 30 04:48:40.412705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 30 04:48:40.412705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 30 04:48:40.412705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 30 04:48:40.412705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 639/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Jul 30 04:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:40.419417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d3470-da49-461d-b8a2-3d748c9bfe75 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:40.469313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:40.519580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d3470-da49-461d-b8a2-3d748c9bfe75 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:40.552665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9ccaf-3e17-48de-805e-6d425ffd58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:40.553280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9ccaf-3e17-48de-805e-6d425ffd58c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:40.553560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9ccaf-3e17-48de-805e-6d425ffd58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:40.553833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9ccaf-3e17-48de-805e-6d425ffd58c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:40.554612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9ccaf-3e17-48de-805e-6d425ffd58c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:40.554919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9ccaf-3e17-48de-805e-6d425ffd58c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:40.556341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9ccaf-3e17-48de-805e-6d425ffd58c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:40.568675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 647/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Jul 30 04:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:40.575102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34c2ee-fe85-4f91-a672-d70a4f139244 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:40.674490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34c2ee-fe85-4f91-a672-d70a4f139244 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:40.706872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665de5c9-7104-4a43-88ef-f5eb5f9fdde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:40.707483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665de5c9-7104-4a43-88ef-f5eb5f9fdde7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:40.707785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665de5c9-7104-4a43-88ef-f5eb5f9fdde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:40.708015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665de5c9-7104-4a43-88ef-f5eb5f9fdde7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:40.708801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665de5c9-7104-4a43-88ef-f5eb5f9fdde7 None admin] RBAC: Adding query filter params (role.id=c09e805a3a134c229ec9d02fe9bea8cb, scope.project.id=2bfc8c66b48b4981832f2c4b940df124, user.id=21b7c64636bd47e6b9a486b6cdaab4b6) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:40.709105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665de5c9-7104-4a43-88ef-f5eb5f9fdde7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:40.710566 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665de5c9-7104-4a43-88ef-f5eb5f9fdde7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:40.717667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 640/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Jul 30 04:48:40 2022] GET /identity/v3/role_assignments?user.id=21b7c64636bd47e6b9a486b6cdaab4b6&scope.project.id=2bfc8c66b48b4981832f2c4b940df124&role.id=c09e805a3a134c229ec9d02fe9bea8cb => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:42.770411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329cdad-37df-4b30-8cee-6ec35c709778 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:42.770982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329cdad-37df-4b30-8cee-6ec35c709778 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:42.770982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329cdad-37df-4b30-8cee-6ec35c709778 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:42.772326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 648/1288] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:42.777981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a4580-f6f9-4436-b460-7ee7c4bf9520 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:42.778418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a4580-f6f9-4436-b460-7ee7c4bf9520 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:42.778556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a4580-f6f9-4436-b460-7ee7c4bf9520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:42.797305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e44a4580-f6f9-4436-b460-7ee7c4bf9520 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:42.798515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e44a4580-f6f9-4436-b460-7ee7c4bf9520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:42.828167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:42.865306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 641/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:42.888198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329cdad-37df-4b30-8cee-6ec35c709778 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:42.989952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329cdad-37df-4b30-8cee-6ec35c709778 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:43.001424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:43.025745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd4a2a-eb2a-4bb9-aadd-852953a18634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:43.026379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd4a2a-eb2a-4bb9-aadd-852953a18634 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:43.026691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd4a2a-eb2a-4bb9-aadd-852953a18634 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:43.027044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd4a2a-eb2a-4bb9-aadd-852953a18634 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:43.027664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd4a2a-eb2a-4bb9-aadd-852953a18634 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:43.029012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd4a2a-eb2a-4bb9-aadd-852953a18634 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-11dd4a2a-eb2a-4bb9-aadd-852953a18634 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:43.037619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:43.038885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:43.039776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 649/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:48:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:43.044917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a4580-f6f9-4436-b460-7ee7c4bf9520 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:43.140375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a4580-f6f9-4436-b460-7ee7c4bf9520 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:43.172582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa21cc6-1177-4bf7-a712-0dbdd91f0450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:43.173181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa21cc6-1177-4bf7-a712-0dbdd91f0450 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:43.173454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa21cc6-1177-4bf7-a712-0dbdd91f0450 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:43.173722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa21cc6-1177-4bf7-a712-0dbdd91f0450 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:43.174380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa21cc6-1177-4bf7-a712-0dbdd91f0450 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:43.174704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa21cc6-1177-4bf7-a712-0dbdd91f0450 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:43.176047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa21cc6-1177-4bf7-a712-0dbdd91f0450 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:43.180684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 642/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Sat Jul 30 04:48:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:43.186968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd4a2a-eb2a-4bb9-aadd-852953a18634 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:43.285918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd4a2a-eb2a-4bb9-aadd-852953a18634 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:43.319001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb0dbd-af16-4228-aefb-0ce63b88a0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:43.319606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb0dbd-af16-4228-aefb-0ce63b88a0ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:43.319888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb0dbd-af16-4228-aefb-0ce63b88a0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:43.320159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb0dbd-af16-4228-aefb-0ce63b88a0ee None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:43.320825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb0dbd-af16-4228-aefb-0ce63b88a0ee None admin] RBAC: Adding query filter params (type=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:43.321126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb0dbd-af16-4228-aefb-0ce63b88a0ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:43.322500 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb0dbd-af16-4228-aefb-0ce63b88a0ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:43.327033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 650/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Sat Jul 30 04:48:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:45.345601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7a9a4-f05f-4a50-b899-87c2cbe12cee None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:45.346200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7a9a4-f05f-4a50-b899-87c2cbe12cee None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:45.346200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7a9a4-f05f-4a50-b899-87c2cbe12cee None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:45.347443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 643/1293] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:45.352468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901989c4-b501-4e6a-a7ab-47048bd412a0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:45.352872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901989c4-b501-4e6a-a7ab-47048bd412a0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:45.353018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901989c4-b501-4e6a-a7ab-47048bd412a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:45.371357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-901989c4-b501-4e6a-a7ab-47048bd412a0 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:45.372606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-901989c4-b501-4e6a-a7ab-47048bd412a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:45.402815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:45.439832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 651/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:45.462665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7a9a4-f05f-4a50-b899-87c2cbe12cee None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:45.493831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:45.564710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7a9a4-f05f-4a50-b899-87c2cbe12cee None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:45.598256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acff86-9505-42a4-8463-ef57ccc7988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:45.598820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acff86-9505-42a4-8463-ef57ccc7988c None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:45.599094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acff86-9505-42a4-8463-ef57ccc7988c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:45.599359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acff86-9505-42a4-8463-ef57ccc7988c None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:45.600036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acff86-9505-42a4-8463-ef57ccc7988c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:45.601377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acff86-9505-42a4-8463-ef57ccc7988c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:45.615274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 644/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Jul 30 04:48:45 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:48:47.645118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592b779-fbe2-4688-b51f-84c6e5f92817 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:47.645667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592b779-fbe2-4688-b51f-84c6e5f92817 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:47.645667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592b779-fbe2-4688-b51f-84c6e5f92817 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:47.647042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 652/1296] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:47.652194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dea135-4135-42aa-9193-030b650a0c1f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:47.652643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dea135-4135-42aa-9193-030b650a0c1f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:47.652744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dea135-4135-42aa-9193-030b650a0c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:47.670521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-19dea135-4135-42aa-9193-030b650a0c1f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:47.671704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-19dea135-4135-42aa-9193-030b650a0c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:47.751445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 645/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:47 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:47.776751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592b779-fbe2-4688-b51f-84c6e5f92817 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:47.877709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592b779-fbe2-4688-b51f-84c6e5f92817 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:47.910448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a37a9-61af-47d0-b7e1-26aec50fe946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:47.911032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a37a9-61af-47d0-b7e1-26aec50fe946 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:47.911308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a37a9-61af-47d0-b7e1-26aec50fe946 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:47.911577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a37a9-61af-47d0-b7e1-26aec50fe946 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:47.912186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a37a9-61af-47d0-b7e1-26aec50fe946 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:47.913536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a37a9-61af-47d0-b7e1-26aec50fe946 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2c2a37a9-61af-47d0-b7e1-26aec50fe946 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:47.922719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:47.923992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:47.925142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 653/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:48:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:47.929975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dea135-4135-42aa-9193-030b650a0c1f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:47.948712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:48.029081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dea135-4135-42aa-9193-030b650a0c1f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:48.062563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983caa0-cf8a-4935-bf10-ece7834f6f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:48.063173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983caa0-cf8a-4935-bf10-ece7834f6f15 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:48.063450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983caa0-cf8a-4935-bf10-ece7834f6f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:48.063721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983caa0-cf8a-4935-bf10-ece7834f6f15 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:48.064389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983caa0-cf8a-4935-bf10-ece7834f6f15 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:48.064690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983caa0-cf8a-4935-bf10-ece7834f6f15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:48.066050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983caa0-cf8a-4935-bf10-ece7834f6f15 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:48.071061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 646/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Sat Jul 30 04:48:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:48.077371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a37a9-61af-47d0-b7e1-26aec50fe946 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:48.109258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:48.177058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a37a9-61af-47d0-b7e1-26aec50fe946 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:48.209437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77d692-8182-4a61-a84a-ea0867b4fb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:48.210042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77d692-8182-4a61-a84a-ea0867b4fb57 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:48.210320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77d692-8182-4a61-a84a-ea0867b4fb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:48.210622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77d692-8182-4a61-a84a-ea0867b4fb57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:48.211369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77d692-8182-4a61-a84a-ea0867b4fb57 None admin] RBAC: Adding query filter params (interface=public, service_id=72e826a128494606a960e2c911c40656, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:48.211676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77d692-8182-4a61-a84a-ea0867b4fb57 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:48.213027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77d692-8182-4a61-a84a-ea0867b4fb57 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:48.218705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 654/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Jul 30 04:48:48 2022] GET /identity/v3/endpoints?service_id=72e826a128494606a960e2c911c40656&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:48.224234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983caa0-cf8a-4935-bf10-ece7834f6f15 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:48.304538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:48.322808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983caa0-cf8a-4935-bf10-ece7834f6f15 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:48.356620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6fa9e-de84-409f-b42a-70abf19f74d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:48.357233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6fa9e-de84-409f-b42a-70abf19f74d1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:48.357509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6fa9e-de84-409f-b42a-70abf19f74d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:48.357780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6fa9e-de84-409f-b42a-70abf19f74d1 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:48.358423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6fa9e-de84-409f-b42a-70abf19f74d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:48.359837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6fa9e-de84-409f-b42a-70abf19f74d1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:48.366630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 647/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Jul 30 04:48:48 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:48:50.322964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585d1c9-5dfc-4c7c-a5a6-63f68ac4c46e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:50.325394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585d1c9-5dfc-4c7c-a5a6-63f68ac4c46e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:50.325671 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585d1c9-5dfc-4c7c-a5a6-63f68ac4c46e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:50.327072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 655/1302] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:48:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:48:50.332001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da32bc6-683f-4bd0-a589-53b36b705e25 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:50.332402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da32bc6-683f-4bd0-a589-53b36b705e25 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:50.332563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da32bc6-683f-4bd0-a589-53b36b705e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:50.350131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0da32bc6-683f-4bd0-a589-53b36b705e25 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:48:50.351364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0da32bc6-683f-4bd0-a589-53b36b705e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:48:50.433350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 648/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:48:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 30 04:48:50.455785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585d1c9-5dfc-4c7c-a5a6-63f68ac4c46e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:50.551005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:50.558776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585d1c9-5dfc-4c7c-a5a6-63f68ac4c46e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:50.591625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994d1fa-7537-4890-9140-44e32220bae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:50.592193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994d1fa-7537-4890-9140-44e32220bae7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:50.592469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994d1fa-7537-4890-9140-44e32220bae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:50.592734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994d1fa-7537-4890-9140-44e32220bae7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:50.593343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994d1fa-7537-4890-9140-44e32220bae7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:50.594722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994d1fa-7537-4890-9140-44e32220bae7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2994d1fa-7537-4890-9140-44e32220bae7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 30 04:48:50.603242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 30 04:48:50.604565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 30 04:48:50.605448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 656/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Jul 30 04:48:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 30 04:48:50.610625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da32bc6-683f-4bd0-a589-53b36b705e25 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:50.685643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:48:50.708358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da32bc6-683f-4bd0-a589-53b36b705e25 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:50.743668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1abeb-af37-4a32-a48a-0b05a3bbb02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:50.744328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1abeb-af37-4a32-a48a-0b05a3bbb02b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:50.745369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1abeb-af37-4a32-a48a-0b05a3bbb02b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:50.745515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1abeb-af37-4a32-a48a-0b05a3bbb02b None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:50.746154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1abeb-af37-4a32-a48a-0b05a3bbb02b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:48:50.746471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1abeb-af37-4a32-a48a-0b05a3bbb02b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:50.747943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1abeb-af37-4a32-a48a-0b05a3bbb02b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:50.753411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 649/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Sat Jul 30 04:48:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:48:50.759925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994d1fa-7537-4890-9140-44e32220bae7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:48:50.865451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994d1fa-7537-4890-9140-44e32220bae7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:48:50.899700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dab240-9ac0-4be7-af92-8b34233546d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:48:50.900310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dab240-9ac0-4be7-af92-8b34233546d3 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:48:50.900589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dab240-9ac0-4be7-af92-8b34233546d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:48:50.900861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dab240-9ac0-4be7-af92-8b34233546d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:48:50.902209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dab240-9ac0-4be7-af92-8b34233546d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:48:50.903611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dab240-9ac0-4be7-af92-8b34233546d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:48:50.922356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 657/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:48:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:49:00.750630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=30 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:49:00.918667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:49:53.403608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0b441-72bc-4a8e-a4c8-e7345123048b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:53.404652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0b441-72bc-4a8e-a4c8-e7345123048b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:53.405156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0b441-72bc-4a8e-a4c8-e7345123048b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:53.410733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 650/1307] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:49:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:49:53.412076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fd8ef-02d8-45de-9457-c05e3b6ea817 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:53.412492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fd8ef-02d8-45de-9457-c05e3b6ea817 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:53.412697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fd8ef-02d8-45de-9457-c05e3b6ea817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:53.504689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-872fd8ef-02d8-45de-9457-c05e3b6ea817 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:49:53.506311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-872fd8ef-02d8-45de-9457-c05e3b6ea817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:49:53.763579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 658/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:49:53.818038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833212fb-c8ba-45c1-8f74-89ebe852554f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:53.818609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833212fb-c8ba-45c1-8f74-89ebe852554f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:53.819038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833212fb-c8ba-45c1-8f74-89ebe852554f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:53.820675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 651/1309] 10.222.0.20 () {54 vars in 965 bytes} [Sat Jul 30 04:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:49:53.829616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba172b5-f52a-4fdd-a3da-805a48d3d555 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:53.830222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba172b5-f52a-4fdd-a3da-805a48d3d555 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:53.830745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba172b5-f52a-4fdd-a3da-805a48d3d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:53.947102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fba172b5-f52a-4fdd-a3da-805a48d3d555 None None] MFA Rules not processed for user `426be56424504e958402d785a068d2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:49:53.948412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fba172b5-f52a-4fdd-a3da-805a48d3d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:49:53.974361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:49:54.065874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 659/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Sat Jul 30 04:49:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:49:54.066779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833212fb-c8ba-45c1-8f74-89ebe852554f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:49:54.068827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-833212fb-c8ba-45c1-8f74-89ebe852554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:49:54.193665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833212fb-c8ba-45c1-8f74-89ebe852554f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:49:54.278202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093aa36-71bf-45b6-9fba-71588e351b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:49:54.278875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093aa36-71bf-45b6-9fba-71588e351b63 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:54.279153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093aa36-71bf-45b6-9fba-71588e351b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:54.279426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093aa36-71bf-45b6-9fba-71588e351b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:54.281069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3093aa36-71bf-45b6-9fba-71588e351b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:49:54.349174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:49:54.430606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093aa36-71bf-45b6-9fba-71588e351b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:49:54.432891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3093aa36-71bf-45b6-9fba-71588e351b63 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:49:54.456086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 652/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:49:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:49:56.789766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63aaba8-58fa-405e-a84c-d1de4a6f667f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:56.790344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63aaba8-58fa-405e-a84c-d1de4a6f667f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:56.790344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63aaba8-58fa-405e-a84c-d1de4a6f667f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:56.791708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 660/1312] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:49:56.797097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff320c-b07d-4d50-83db-3bc188f8650e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:56.797510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff320c-b07d-4d50-83db-3bc188f8650e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:56.797643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff320c-b07d-4d50-83db-3bc188f8650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:56.868872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-25ff320c-b07d-4d50-83db-3bc188f8650e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:49:56.870085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-25ff320c-b07d-4d50-83db-3bc188f8650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:49:56.977715 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 653/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:49:57.033313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb6465-a8e8-4632-853a-288633f54965 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:57.033738 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb6465-a8e8-4632-853a-288633f54965 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:57.034733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb6465-a8e8-4632-853a-288633f54965 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:57.036132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 661/1314] 10.222.0.20 () {54 vars in 965 bytes} [Sat Jul 30 04:49:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:49:57.042646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4927d27-4850-42b2-9cff-a171f010b824 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:57.042936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4927d27-4850-42b2-9cff-a171f010b824 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:57.043216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4927d27-4850-42b2-9cff-a171f010b824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:57.055005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:49:57.120157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f4927d27-4850-42b2-9cff-a171f010b824 None None] MFA Rules not processed for user `426be56424504e958402d785a068d2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:49:57.121368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f4927d27-4850-42b2-9cff-a171f010b824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:49:57.199845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 654/1315] 10.222.0.20 () {58 vars in 1062 bytes} [Sat Jul 30 04:49:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:49:57.208946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb6465-a8e8-4632-853a-288633f54965 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:49:57.210518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2fb6465-a8e8-4632-853a-288633f54965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:49:57.320217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb6465-a8e8-4632-853a-288633f54965 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:49:57.391343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:49:57.394335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9df33-770e-4d0f-8dce-4094e80c16e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:49:57.395182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9df33-770e-4d0f-8dce-4094e80c16e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:49:57.395508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9df33-770e-4d0f-8dce-4094e80c16e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:49:57.395926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9df33-770e-4d0f-8dce-4094e80c16e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:49:57.397787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0da9df33-770e-4d0f-8dce-4094e80c16e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:49:57.505228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9df33-770e-4d0f-8dce-4094e80c16e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:49:57.506685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9df33-770e-4d0f-8dce-4094e80c16e4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:49:57.524406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 662/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:00.048357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3f2c0-0c7d-4360-b54c-e5ea5dfccb63 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:00.049438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3f2c0-0c7d-4360-b54c-e5ea5dfccb63 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:00.049605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3f2c0-0c7d-4360-b54c-e5ea5dfccb63 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:00.051034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 655/1317] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:00.056102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b239e-5e90-4dd0-9766-1dbbd53838ec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:00.056534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b239e-5e90-4dd0-9766-1dbbd53838ec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:00.056663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b239e-5e90-4dd0-9766-1dbbd53838ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:00.122144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-308b239e-5e90-4dd0-9766-1dbbd53838ec None None] MFA Rules not processed for user `5716eec426c840c79a2d8f3c7ec7ad18`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:00.123415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-308b239e-5e90-4dd0-9766-1dbbd53838ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:00.177409 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:00.223281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 663/1318] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:00.262869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3f2c0-0c7d-4360-b54c-e5ea5dfccb63 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:00.264401 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d1e3f2c0-0c7d-4360-b54c-e5ea5dfccb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:00.368010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3f2c0-0c7d-4360-b54c-e5ea5dfccb63 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:00.370734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:00.439529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18150e9e-6089-4945-891f-464d20e6bbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:00.440127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18150e9e-6089-4945-891f-464d20e6bbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:00.440402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18150e9e-6089-4945-891f-464d20e6bbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:00.440669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18150e9e-6089-4945-891f-464d20e6bbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:00.442274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-18150e9e-6089-4945-891f-464d20e6bbc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:00.548500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18150e9e-6089-4945-891f-464d20e6bbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:00.549992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18150e9e-6089-4945-891f-464d20e6bbc9 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:00.568693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 656/1319] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:03.140660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214f276-33c0-48a4-ad15-74472855eb95 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:03.141182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214f276-33c0-48a4-ad15-74472855eb95 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:03.141278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214f276-33c0-48a4-ad15-74472855eb95 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:03.143668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 664/1320] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:03.148753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c580a-c145-4d77-b270-be98f1ee00b1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:03.149171 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c580a-c145-4d77-b270-be98f1ee00b1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:03.149309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c580a-c145-4d77-b270-be98f1ee00b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:03.189924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:03.216351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-051c580a-c145-4d77-b270-be98f1ee00b1 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:03.217587 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-051c580a-c145-4d77-b270-be98f1ee00b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:03.312274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 657/1321] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:03.351172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214f276-33c0-48a4-ad15-74472855eb95 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:03.352724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6214f276-33c0-48a4-ad15-74472855eb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:03.456049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214f276-33c0-48a4-ad15-74472855eb95 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:03.527157 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341e532-4adb-4a61-bc5b-1949100afdcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:03.527781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341e532-4adb-4a61-bc5b-1949100afdcd service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:03.528040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341e532-4adb-4a61-bc5b-1949100afdcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:03.528310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341e532-4adb-4a61-bc5b-1949100afdcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:03.529919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d341e532-4adb-4a61-bc5b-1949100afdcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:03.538021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:03.638639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341e532-4adb-4a61-bc5b-1949100afdcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:03.639995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341e532-4adb-4a61-bc5b-1949100afdcd service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:03.658207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 665/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:06.310194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73092040-9a0c-4b85-a470-a769e47240e1 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:06.310750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73092040-9a0c-4b85-a470-a769e47240e1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:06.310750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73092040-9a0c-4b85-a470-a769e47240e1 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:06.312083 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 658/1323] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:06.318129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b05637-ce79-44ff-be15-dfa7212e95b7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:06.318556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b05637-ce79-44ff-be15-dfa7212e95b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:06.318840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b05637-ce79-44ff-be15-dfa7212e95b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:06.384930 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a9b05637-ce79-44ff-be15-dfa7212e95b7 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:06.386125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a9b05637-ce79-44ff-be15-dfa7212e95b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:06.445215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:06.488438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 666/1324] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:06.524267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73092040-9a0c-4b85-a470-a769e47240e1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:06.526026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-73092040-9a0c-4b85-a470-a769e47240e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:06.638077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73092040-9a0c-4b85-a470-a769e47240e1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:06.651204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:06.715060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824559e6-5561-4aac-b918-bc68b01ab6ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:06.715759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824559e6-5561-4aac-b918-bc68b01ab6ec service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:06.715961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824559e6-5561-4aac-b918-bc68b01ab6ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:06.716323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824559e6-5561-4aac-b918-bc68b01ab6ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:06.717887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-824559e6-5561-4aac-b918-bc68b01ab6ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:06.828505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824559e6-5561-4aac-b918-bc68b01ab6ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:06.829977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824559e6-5561-4aac-b918-bc68b01ab6ec service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:06.848358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 659/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:08.929916 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9e3dc-b244-4272-8bd0-8828725f59ff None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:08.930507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9e3dc-b244-4272-8bd0-8828725f59ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:08.930507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9e3dc-b244-4272-8bd0-8828725f59ff None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:08.931848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 667/1326] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:08.936957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93468271-58bd-4690-9c8f-237996f02a6f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:08.937409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93468271-58bd-4690-9c8f-237996f02a6f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:08.937518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93468271-58bd-4690-9c8f-237996f02a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:08.987604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:09.010276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-93468271-58bd-4690-9c8f-237996f02a6f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:09.012763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-93468271-58bd-4690-9c8f-237996f02a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:09.117424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 660/1327] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:09.155104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9e3dc-b244-4272-8bd0-8828725f59ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:09.156795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-21d9e3dc-b244-4272-8bd0-8828725f59ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:09.263018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9e3dc-b244-4272-8bd0-8828725f59ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:09.340024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db16e5e-d5b5-4118-a9e2-b967738c21f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:09.340649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db16e5e-d5b5-4118-a9e2-b967738c21f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:09.340924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db16e5e-d5b5-4118-a9e2-b967738c21f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:09.341197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db16e5e-d5b5-4118-a9e2-b967738c21f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:09.342880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9db16e5e-d5b5-4118-a9e2-b967738c21f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:09.361265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:09.458701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db16e5e-d5b5-4118-a9e2-b967738c21f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:09.460158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db16e5e-d5b5-4118-a9e2-b967738c21f1 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:09.478279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 668/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:11.547685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b9b66-be83-48c8-947a-6645e58cbf4f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:11.548270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b9b66-be83-48c8-947a-6645e58cbf4f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:11.548270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b9b66-be83-48c8-947a-6645e58cbf4f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:11.549805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 661/1329] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:11.555845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41777d2a-1203-46a9-b3b9-b6ef2a62ed5b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:11.556386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41777d2a-1203-46a9-b3b9-b6ef2a62ed5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:11.556588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41777d2a-1203-46a9-b3b9-b6ef2a62ed5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:11.623787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-41777d2a-1203-46a9-b3b9-b6ef2a62ed5b None None] MFA Rules not processed for user `5716eec426c840c79a2d8f3c7ec7ad18`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:11.625048 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41777d2a-1203-46a9-b3b9-b6ef2a62ed5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:11.692207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:11.726042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 669/1330] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:50:11.763107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b9b66-be83-48c8-947a-6645e58cbf4f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:11.764696 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-893b9b66-be83-48c8-947a-6645e58cbf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:11.874748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b9b66-be83-48c8-947a-6645e58cbf4f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:11.895202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:11.951959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705620f-1462-4340-9636-b15097eb38d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:11.952575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705620f-1462-4340-9636-b15097eb38d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:11.952854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705620f-1462-4340-9636-b15097eb38d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:11.953141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705620f-1462-4340-9636-b15097eb38d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:11.955432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2705620f-1462-4340-9636-b15097eb38d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:12.069798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705620f-1462-4340-9636-b15097eb38d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:12.071280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705620f-1462-4340-9636-b15097eb38d7 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:12.090381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 662/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:14.355193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf0764-d8ec-40e5-9fcb-d4c4ebbb3b2c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:14.355736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf0764-d8ec-40e5-9fcb-d4c4ebbb3b2c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:14.355736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf0764-d8ec-40e5-9fcb-d4c4ebbb3b2c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:14.357013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 670/1332] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:14.362535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf9056-09b8-4006-bea6-b0e782118068 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:14.362921 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf9056-09b8-4006-bea6-b0e782118068 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:14.363088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf9056-09b8-4006-bea6-b0e782118068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:14.426361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:14.429584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-84bf9056-09b8-4006-bea6-b0e782118068 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:14.431201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-84bf9056-09b8-4006-bea6-b0e782118068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:14.525984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 663/1333] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:14.562393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf0764-d8ec-40e5-9fcb-d4c4ebbb3b2c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:14.564070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-96cf0764-d8ec-40e5-9fcb-d4c4ebbb3b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:14.666722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf0764-d8ec-40e5-9fcb-d4c4ebbb3b2c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:14.738692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3014e-49d9-4ab2-81c6-7b1946aaffb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:14.739304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3014e-49d9-4ab2-81c6-7b1946aaffb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:14.739586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3014e-49d9-4ab2-81c6-7b1946aaffb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:14.739863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3014e-49d9-4ab2-81c6-7b1946aaffb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:14.741497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-14b3014e-49d9-4ab2-81c6-7b1946aaffb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:14.768456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:14.851605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3014e-49d9-4ab2-81c6-7b1946aaffb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:14.853059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3014e-49d9-4ab2-81c6-7b1946aaffb3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:14.871278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 671/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:16.938336 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c182a20-5c3e-4e2b-aadc-0dcb651360ba None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:16.938939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c182a20-5c3e-4e2b-aadc-0dcb651360ba None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:16.938939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c182a20-5c3e-4e2b-aadc-0dcb651360ba None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:16.940317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 664/1335] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:16.945389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c49b5-fbb6-4192-83c5-c9cd9af65ff1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:16.945791 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c49b5-fbb6-4192-83c5-c9cd9af65ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:16.945942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c49b5-fbb6-4192-83c5-c9cd9af65ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:17.011457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-783c49b5-fbb6-4192-83c5-c9cd9af65ff1 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:17.012664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-783c49b5-fbb6-4192-83c5-c9cd9af65ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:17.088170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:17.111716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 672/1336] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:17.152143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c182a20-5c3e-4e2b-aadc-0dcb651360ba None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:17.153686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4c182a20-5c3e-4e2b-aadc-0dcb651360ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:17.260040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c182a20-5c3e-4e2b-aadc-0dcb651360ba None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:17.288655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:17.345755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88590a0-f826-47de-ae9f-3fa3b30e634e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:17.346375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88590a0-f826-47de-ae9f-3fa3b30e634e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:17.346684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88590a0-f826-47de-ae9f-3fa3b30e634e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:17.346957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88590a0-f826-47de-ae9f-3fa3b30e634e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:17.348594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a88590a0-f826-47de-ae9f-3fa3b30e634e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:17.457584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88590a0-f826-47de-ae9f-3fa3b30e634e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:17.459041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88590a0-f826-47de-ae9f-3fa3b30e634e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:17.477398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 665/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:19.820616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983920d-e31f-4c9c-abb1-48f202052007 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:19.821216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983920d-e31f-4c9c-abb1-48f202052007 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:19.821216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983920d-e31f-4c9c-abb1-48f202052007 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:19.822588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 673/1338] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:19.828152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443cf9e-4969-4d36-889e-b1eed17c5664 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:19.828600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443cf9e-4969-4d36-889e-b1eed17c5664 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:19.828878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443cf9e-4969-4d36-889e-b1eed17c5664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:19.893668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a443cf9e-4969-4d36-889e-b1eed17c5664 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:19.894890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a443cf9e-4969-4d36-889e-b1eed17c5664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:19.903103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:19.991852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 666/1339] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:20.028497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983920d-e31f-4c9c-abb1-48f202052007 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:20.030055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8983920d-e31f-4c9c-abb1-48f202052007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:20.133239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983920d-e31f-4c9c-abb1-48f202052007 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:20.206239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a7b48-27cf-422d-aae4-5c123a0b304f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:20.206878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a7b48-27cf-422d-aae4-5c123a0b304f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:20.207154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a7b48-27cf-422d-aae4-5c123a0b304f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:20.207423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a7b48-27cf-422d-aae4-5c123a0b304f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:20.209128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e72a7b48-27cf-422d-aae4-5c123a0b304f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:20.243607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:20.317042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a7b48-27cf-422d-aae4-5c123a0b304f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:20.318634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a7b48-27cf-422d-aae4-5c123a0b304f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:20.336873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 674/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:24.741245 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344006b-09aa-4df4-8f81-609822c8f7e1 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:24.741812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344006b-09aa-4df4-8f81-609822c8f7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:24.741812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344006b-09aa-4df4-8f81-609822c8f7e1 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:24.743189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 667/1341] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:24.748384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e9d64-5d4f-483f-ac68-17734b80a72a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:24.748807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e9d64-5d4f-483f-ac68-17734b80a72a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:24.749026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e9d64-5d4f-483f-ac68-17734b80a72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:24.818410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-399e9d64-5d4f-483f-ac68-17734b80a72a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:24.819666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-399e9d64-5d4f-483f-ac68-17734b80a72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:24.909634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:24.922457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 675/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:24.962083 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344006b-09aa-4df4-8f81-609822c8f7e1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:24.963669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c344006b-09aa-4df4-8f81-609822c8f7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:25.072733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344006b-09aa-4df4-8f81-609822c8f7e1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:25.112742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:25.152360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce755b07-03db-4f3c-8e91-89cf6d7a9fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:25.152978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce755b07-03db-4f3c-8e91-89cf6d7a9fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:25.153258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce755b07-03db-4f3c-8e91-89cf6d7a9fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:25.153532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce755b07-03db-4f3c-8e91-89cf6d7a9fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:25.155241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ce755b07-03db-4f3c-8e91-89cf6d7a9fc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:25.269116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce755b07-03db-4f3c-8e91-89cf6d7a9fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:25.270574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce755b07-03db-4f3c-8e91-89cf6d7a9fc4 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:25.288945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 668/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:27.959723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7a9ca-71d1-4819-8c6d-5649930461f0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:27.960271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7a9ca-71d1-4819-8c6d-5649930461f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:27.960271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7a9ca-71d1-4819-8c6d-5649930461f0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:27.961629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 676/1344] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:27.967417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e733b-9fcb-46f1-84c7-d187a5b0d878 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:27.967963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e733b-9fcb-46f1-84c7-d187a5b0d878 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:27.968167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e733b-9fcb-46f1-84c7-d187a5b0d878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:28.043905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-256e733b-9fcb-46f1-84c7-d187a5b0d878 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:28.045264 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-256e733b-9fcb-46f1-84c7-d187a5b0d878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:28.064540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:28.145981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 669/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:28.183857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7a9ca-71d1-4819-8c6d-5649930461f0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:28.185401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20a7a9ca-71d1-4819-8c6d-5649930461f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:28.293785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7a9ca-71d1-4819-8c6d-5649930461f0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:28.371043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c2743-2bdc-4d25-a1a7-bceccff1c3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:28.371589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2743-2bdc-4d25-a1a7-bceccff1c3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:28.371871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2743-2bdc-4d25-a1a7-bceccff1c3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:28.372147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2743-2bdc-4d25-a1a7-bceccff1c3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:28.374332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-701c2743-2bdc-4d25-a1a7-bceccff1c3d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:28.420275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:28.488976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c2743-2bdc-4d25-a1a7-bceccff1c3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:28.490418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c2743-2bdc-4d25-a1a7-bceccff1c3d8 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:28.510191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 677/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:32.451444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998e0b9-88a5-4c26-8b0e-5c1c11851447 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:32.451969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998e0b9-88a5-4c26-8b0e-5c1c11851447 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:32.451969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998e0b9-88a5-4c26-8b0e-5c1c11851447 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:32.453361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 670/1347] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:32.458615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73153b04-a416-41ef-b55e-49c206d8a9d8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:32.459042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73153b04-a416-41ef-b55e-49c206d8a9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:32.459137 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73153b04-a416-41ef-b55e-49c206d8a9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:32.527086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-73153b04-a416-41ef-b55e-49c206d8a9d8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:32.528317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73153b04-a416-41ef-b55e-49c206d8a9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:32.625234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:32.630753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 678/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:32.666510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998e0b9-88a5-4c26-8b0e-5c1c11851447 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:32.668165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b998e0b9-88a5-4c26-8b0e-5c1c11851447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:32.796512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998e0b9-88a5-4c26-8b0e-5c1c11851447 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:32.852683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:32.882523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a4138-ad9d-4393-8df6-7487b61210b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:32.883143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a4138-ad9d-4393-8df6-7487b61210b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:32.883417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a4138-ad9d-4393-8df6-7487b61210b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:32.883683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a4138-ad9d-4393-8df6-7487b61210b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:32.885309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3a1a4138-ad9d-4393-8df6-7487b61210b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:33.004023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a4138-ad9d-4393-8df6-7487b61210b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:33.005494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a4138-ad9d-4393-8df6-7487b61210b8 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:33.025787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 671/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:37.929362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36447e6-6efc-4c0c-a7fe-2683ebad41ac None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:37.929798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36447e6-6efc-4c0c-a7fe-2683ebad41ac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:37.929910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36447e6-6efc-4c0c-a7fe-2683ebad41ac None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:37.932871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 679/1350] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 04:50:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:37.938328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba812a-6107-4ed1-b861-bd80fae1c135 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:37.938791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba812a-6107-4ed1-b861-bd80fae1c135 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:37.939136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba812a-6107-4ed1-b861-bd80fae1c135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:38.061203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-25ba812a-6107-4ed1-b861-bd80fae1c135 None None] MFA Rules not processed for user `21b7c64636bd47e6b9a486b6cdaab4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:38.063763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-25ba812a-6107-4ed1-b861-bd80fae1c135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:38.095627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:38.209661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 672/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 04:50:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:42.423302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c5cd3-d1ac-4db6-ac1a-fe21d3c3cf8f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:42.424247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c5cd3-d1ac-4db6-ac1a-fe21d3c3cf8f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:42.424748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c5cd3-d1ac-4db6-ac1a-fe21d3c3cf8f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:42.426652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 680/1352] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 04:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:42.431893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0846b-73e5-4ea8-b805-48b2e39240ff None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:42.432372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0846b-73e5-4ea8-b805-48b2e39240ff None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:42.432657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0846b-73e5-4ea8-b805-48b2e39240ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:42.511525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-05f0846b-73e5-4ea8-b805-48b2e39240ff None None] MFA Rules not processed for user `21b7c64636bd47e6b9a486b6cdaab4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:42.512933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-05f0846b-73e5-4ea8-b805-48b2e39240ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:42.587457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:42.630083 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 673/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 04:50:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:43.875575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658dfa8-5890-417f-8fcf-f3b55f489a9e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:43.876102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658dfa8-5890-417f-8fcf-f3b55f489a9e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:43.876102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658dfa8-5890-417f-8fcf-f3b55f489a9e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:43.877606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 681/1354] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 04:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:43.882825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a76953-c3c3-48f4-9b1f-3c52aca28e4a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:43.883217 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a76953-c3c3-48f4-9b1f-3c52aca28e4a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:43.883386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a76953-c3c3-48f4-9b1f-3c52aca28e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:43.961206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e8a76953-c3c3-48f4-9b1f-3c52aca28e4a None None] MFA Rules not processed for user `21b7c64636bd47e6b9a486b6cdaab4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:43.962749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e8a76953-c3c3-48f4-9b1f-3c52aca28e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:44.076705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 674/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 04:50:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:50:47.479168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eb3b4-be44-40ac-b60f-aa6122b1e110 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:47.480761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eb3b4-be44-40ac-b60f-aa6122b1e110 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:47.482031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eb3b4-be44-40ac-b60f-aa6122b1e110 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:47.483844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 682/1356] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:47.489359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba45e8c-3618-4f3f-888c-b4d38661b836 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:47.490130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba45e8c-3618-4f3f-888c-b4d38661b836 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:47.490577 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba45e8c-3618-4f3f-888c-b4d38661b836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:47.544028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:47.585678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6ba45e8c-3618-4f3f-888c-b4d38661b836 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:47.587470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6ba45e8c-3618-4f3f-888c-b4d38661b836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:47.697746 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 675/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:47.751833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c74b5-fc53-4d5b-8fd8-127d231d9d62 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:47.752569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c74b5-fc53-4d5b-8fd8-127d231d9d62 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:47.753082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c74b5-fc53-4d5b-8fd8-127d231d9d62 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:47.756051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 683/1358] 10.222.0.20 () {54 vars in 962 bytes} [Sat Jul 30 04:50:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:47.765522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc6a28-255e-46de-bfc1-ea2263cad995 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:47.766251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc6a28-255e-46de-bfc1-ea2263cad995 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:47.766251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc6a28-255e-46de-bfc1-ea2263cad995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:47.849710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-50cc6a28-255e-46de-bfc1-ea2263cad995 None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:47.850841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-50cc6a28-255e-46de-bfc1-ea2263cad995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:47.903962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:47.976655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 676/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Sat Jul 30 04:50:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:47.984429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c74b5-fc53-4d5b-8fd8-127d231d9d62 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:47.986230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e72c74b5-fc53-4d5b-8fd8-127d231d9d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:48.150595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c74b5-fc53-4d5b-8fd8-127d231d9d62 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:48.230973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:48.278071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d104f54-2635-4794-b5b6-67494b818f50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:48.278816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d104f54-2635-4794-b5b6-67494b818f50 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:48.279209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d104f54-2635-4794-b5b6-67494b818f50 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:48.279583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d104f54-2635-4794-b5b6-67494b818f50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:48.281318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d104f54-2635-4794-b5b6-67494b818f50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:48.375355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d104f54-2635-4794-b5b6-67494b818f50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:48.377119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d104f54-2635-4794-b5b6-67494b818f50 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:48.397129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 684/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:50.628305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc30c0-12f4-4800-8395-39a108a6b29f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:50.628829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc30c0-12f4-4800-8395-39a108a6b29f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:50.628829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc30c0-12f4-4800-8395-39a108a6b29f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:50.630574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 677/1361] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:50.635403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec37de-99d4-4a2a-b6f1-c41e880fc811 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:50.636338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec37de-99d4-4a2a-b6f1-c41e880fc811 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:50.636584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec37de-99d4-4a2a-b6f1-c41e880fc811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:50.717845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d1ec37de-99d4-4a2a-b6f1-c41e880fc811 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:50.717845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1ec37de-99d4-4a2a-b6f1-c41e880fc811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:50.765173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:50.838491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 685/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:50.878332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc30c0-12f4-4800-8395-39a108a6b29f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:50.880179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-56bc30c0-12f4-4800-8395-39a108a6b29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:51.055303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc30c0-12f4-4800-8395-39a108a6b29f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:51.066021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:51.211365 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ca1ef-a890-44c3-a19b-3406c68942b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:51.212097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca1ef-a890-44c3-a19b-3406c68942b3 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:51.212473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca1ef-a890-44c3-a19b-3406c68942b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:51.212841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca1ef-a890-44c3-a19b-3406c68942b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:51.215356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-610ca1ef-a890-44c3-a19b-3406c68942b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:51.310532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ca1ef-a890-44c3-a19b-3406c68942b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:51.312187 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ca1ef-a890-44c3-a19b-3406c68942b3 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:51.335327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 678/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:53.729622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dd603-9836-4db6-a26e-c194e5418d4c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:53.730280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dd603-9836-4db6-a26e-c194e5418d4c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:53.730280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07dd603-9836-4db6-a26e-c194e5418d4c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:53.732097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 686/1364] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:53.737792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce02dc9-b4bb-40b5-bb5b-0cde6d4d4154 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:53.738630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce02dc9-b4bb-40b5-bb5b-0cde6d4d4154 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:53.738904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce02dc9-b4bb-40b5-bb5b-0cde6d4d4154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:53.755775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:53.817936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0ce02dc9-b4bb-40b5-bb5b-0cde6d4d4154 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:53.819168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0ce02dc9-b4bb-40b5-bb5b-0cde6d4d4154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:53.933038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 679/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:53.981197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c78c8-02ff-4208-946c-3c15dbfa36e7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:53.981404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c78c8-02ff-4208-946c-3c15dbfa36e7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:53.981857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c78c8-02ff-4208-946c-3c15dbfa36e7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:53.983227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 687/1366] 10.222.0.20 () {54 vars in 962 bytes} [Sat Jul 30 04:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:53.989619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebed74a-c649-46fc-881e-ae99b4c1853a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:53.989986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebed74a-c649-46fc-881e-ae99b4c1853a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:53.990352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebed74a-c649-46fc-881e-ae99b4c1853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:54.071918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7ebed74a-c649-46fc-881e-ae99b4c1853a None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:54.073198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7ebed74a-c649-46fc-881e-ae99b4c1853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:54.113027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:54.203735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 680/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Sat Jul 30 04:50:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:54.211582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c78c8-02ff-4208-946c-3c15dbfa36e7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:54.213298 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b8c78c8-02ff-4208-946c-3c15dbfa36e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:54.376845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c78c8-02ff-4208-946c-3c15dbfa36e7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:54.398883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:54.497532 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf9a78-c571-4a6d-ba8c-5c4b62999a20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:54.498164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf9a78-c571-4a6d-ba8c-5c4b62999a20 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:54.498459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf9a78-c571-4a6d-ba8c-5c4b62999a20 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:54.498748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf9a78-c571-4a6d-ba8c-5c4b62999a20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:54.500381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bcbf9a78-c571-4a6d-ba8c-5c4b62999a20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:54.597421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf9a78-c571-4a6d-ba8c-5c4b62999a20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:54.598961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf9a78-c571-4a6d-ba8c-5c4b62999a20 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:54.618795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 688/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:55.622617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479387d2-4fb2-4fad-bce2-21602c349ad7 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:55.623156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479387d2-4fb2-4fad-bce2-21602c349ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:55.623156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479387d2-4fb2-4fad-bce2-21602c349ad7 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:55.625697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 681/1369] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:55.630274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97f23c-8290-43d4-a03f-cee8b840f47d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:55.630730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97f23c-8290-43d4-a03f-cee8b840f47d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:55.630944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97f23c-8290-43d4-a03f-cee8b840f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:55.666614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:55.703861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1f97f23c-8290-43d4-a03f-cee8b840f47d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:55.705065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f97f23c-8290-43d4-a03f-cee8b840f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:55.822500 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 689/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:56.158005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a78e8-f196-4ec4-94da-a99bafd9def4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:56.158005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a78e8-f196-4ec4-94da-a99bafd9def4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:56.158005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a78e8-f196-4ec4-94da-a99bafd9def4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:56.160718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 682/1371] 10.222.0.20 () {54 vars in 964 bytes} [Sat Jul 30 04:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:56.164144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd4ee1-4624-4227-8df7-815753448731 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:56.164437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd4ee1-4624-4227-8df7-815753448731 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:56.164712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd4ee1-4624-4227-8df7-815753448731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:56.246079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-37cd4ee1-4624-4227-8df7-815753448731 None None] MFA Rules not processed for user `a0d2552bd0e244dea55eb1eecf471fef`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:56.247329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-37cd4ee1-4624-4227-8df7-815753448731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:56.296945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:56.330502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 690/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Sat Jul 30 04:50:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:56.339222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a78e8-f196-4ec4-94da-a99bafd9def4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:56.340738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c18a78e8-f196-4ec4-94da-a99bafd9def4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:56.456133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a78e8-f196-4ec4-94da-a99bafd9def4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:56.502386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:56.539398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d86eb-0db0-46f9-8b1a-29fd93f938a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:56.540014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d86eb-0db0-46f9-8b1a-29fd93f938a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:56.540284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d86eb-0db0-46f9-8b1a-29fd93f938a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:56.540549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d86eb-0db0-46f9-8b1a-29fd93f938a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:56.542752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e25d86eb-0db0-46f9-8b1a-29fd93f938a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:56.662774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d86eb-0db0-46f9-8b1a-29fd93f938a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:56.664136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d86eb-0db0-46f9-8b1a-29fd93f938a9 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:56.686722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 683/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:56.709973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993dcd3-cd5c-44c8-b505-effd084f672b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:56.710481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993dcd3-cd5c-44c8-b505-effd084f672b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:56.710481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993dcd3-cd5c-44c8-b505-effd084f672b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:56.711967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 691/1374] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:56.718382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3f3e4-f4f0-415a-be97-72060c26c8be None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:56.718679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3f3e4-f4f0-415a-be97-72060c26c8be None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:56.718946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3f3e4-f4f0-415a-be97-72060c26c8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:56.792281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1fd3f3e4-f4f0-415a-be97-72060c26c8be None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:56.793527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1fd3f3e4-f4f0-415a-be97-72060c26c8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:56.816464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:56.899612 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 684/1375] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:56.945262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993dcd3-cd5c-44c8-b505-effd084f672b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:56.948684 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e993dcd3-cd5c-44c8-b505-effd084f672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:57.151412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993dcd3-cd5c-44c8-b505-effd084f672b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:57.206091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:57.268973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87592e-ddbf-4882-8472-79e8ed60cd9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:57.269598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87592e-ddbf-4882-8472-79e8ed60cd9a service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:57.269877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87592e-ddbf-4882-8472-79e8ed60cd9a service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:57.270148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87592e-ddbf-4882-8472-79e8ed60cd9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:57.271792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e87592e-ddbf-4882-8472-79e8ed60cd9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:57.362858 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87592e-ddbf-4882-8472-79e8ed60cd9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:50:57.364447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87592e-ddbf-4882-8472-79e8ed60cd9a service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:50:57.384383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 692/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:50:59.445343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f689f25-805e-4b1a-ace8-4303415f176e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:59.445924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f689f25-805e-4b1a-ace8-4303415f176e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:59.445924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f689f25-805e-4b1a-ace8-4303415f176e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:59.448035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 685/1377] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:50:59.453205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5876d5-98e2-4f1c-ba72-17cbfec8d016 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:59.453668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5876d5-98e2-4f1c-ba72-17cbfec8d016 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:59.453778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5876d5-98e2-4f1c-ba72-17cbfec8d016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:59.529206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9e5876d5-98e2-4f1c-ba72-17cbfec8d016 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:50:59.530420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9e5876d5-98e2-4f1c-ba72-17cbfec8d016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:59.537602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:59.653388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 693/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:50:59.696887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f689f25-805e-4b1a-ace8-4303415f176e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:50:59.699362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5f689f25-805e-4b1a-ace8-4303415f176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:50:59.851167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f689f25-805e-4b1a-ace8-4303415f176e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:50:59.854898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:50:59.970935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8675fa76-1006-4503-aaea-95b393c626da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:50:59.971561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675fa76-1006-4503-aaea-95b393c626da service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:50:59.971915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675fa76-1006-4503-aaea-95b393c626da service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:50:59.972192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8675fa76-1006-4503-aaea-95b393c626da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:50:59.973833 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8675fa76-1006-4503-aaea-95b393c626da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:00.080888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675fa76-1006-4503-aaea-95b393c626da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:00.082514 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8675fa76-1006-4503-aaea-95b393c626da service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:00.109927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 686/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:50:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:02.204136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0b27d-91bc-402c-8456-1bc2654ba8a3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:02.204685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0b27d-91bc-402c-8456-1bc2654ba8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:02.204685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0b27d-91bc-402c-8456-1bc2654ba8a3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:02.206031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 694/1380] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:02.212940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355d2ea-0edc-42ad-af79-5965100b348e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:02.213549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355d2ea-0edc-42ad-af79-5965100b348e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:02.213549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355d2ea-0edc-42ad-af79-5965100b348e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:02.227371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:02.293263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2355d2ea-0edc-42ad-af79-5965100b348e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:02.294509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2355d2ea-0edc-42ad-af79-5965100b348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:02.401065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 687/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:02.444575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0b27d-91bc-402c-8456-1bc2654ba8a3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:02.446325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76f0b27d-91bc-402c-8456-1bc2654ba8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:02.601227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:02.613087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0b27d-91bc-402c-8456-1bc2654ba8a3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:02.741531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33fd6e-5ac5-498c-bd6d-8727c5f1ce15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:02.742610 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33fd6e-5ac5-498c-bd6d-8727c5f1ce15 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:02.742958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33fd6e-5ac5-498c-bd6d-8727c5f1ce15 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:02.743430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33fd6e-5ac5-498c-bd6d-8727c5f1ce15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:02.746128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-de33fd6e-5ac5-498c-bd6d-8727c5f1ce15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:02.848487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33fd6e-5ac5-498c-bd6d-8727c5f1ce15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:02.850251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33fd6e-5ac5-498c-bd6d-8727c5f1ce15 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:02.862754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:02.877765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 695/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:05.202479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4b4cc-dacf-4127-8115-34bf059b0c0d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:05.203090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4b4cc-dacf-4127-8115-34bf059b0c0d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:05.203090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4b4cc-dacf-4127-8115-34bf059b0c0d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:05.204374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 688/1383] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:05.210599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625fed4-c293-49a8-b63c-66d1ce015a00 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:05.211017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625fed4-c293-49a8-b63c-66d1ce015a00 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:05.211147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625fed4-c293-49a8-b63c-66d1ce015a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:05.288227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7625fed4-c293-49a8-b63c-66d1ce015a00 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:05.289987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7625fed4-c293-49a8-b63c-66d1ce015a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:05.400182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 696/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:51:05.454040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4b4cc-dacf-4127-8115-34bf059b0c0d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:05.456065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a3d4b4cc-dacf-4127-8115-34bf059b0c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:05.561461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:05.626418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4b4cc-dacf-4127-8115-34bf059b0c0d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:05.744191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce368c2-2a83-4654-ad77-f354e038a689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:05.744811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce368c2-2a83-4654-ad77-f354e038a689 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:05.745096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce368c2-2a83-4654-ad77-f354e038a689 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:05.745368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce368c2-2a83-4654-ad77-f354e038a689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:05.747037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dce368c2-2a83-4654-ad77-f354e038a689 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:05.798112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:05.840727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce368c2-2a83-4654-ad77-f354e038a689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:05.842290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce368c2-2a83-4654-ad77-f354e038a689 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:05.865082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 689/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:08.383303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b38749-9adf-4549-b88b-7016ae64507c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:08.383984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b38749-9adf-4549-b88b-7016ae64507c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:08.383984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b38749-9adf-4549-b88b-7016ae64507c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:08.385943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 697/1386] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:08.391431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9a075-9eef-4006-9fca-653da9fb0c81 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:08.391901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9a075-9eef-4006-9fca-653da9fb0c81 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:08.392011 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9a075-9eef-4006-9fca-653da9fb0c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:08.459872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-18a9a075-9eef-4006-9fca-653da9fb0c81 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:08.461072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-18a9a075-9eef-4006-9fca-653da9fb0c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:08.573308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 690/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:08.628155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b38749-9adf-4549-b88b-7016ae64507c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:08.630351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41b38749-9adf-4549-b88b-7016ae64507c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:08.728529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:08.811647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b38749-9adf-4549-b88b-7016ae64507c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:08.943181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e5baf-7e00-461a-bdab-3705564defcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:08.943654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5baf-7e00-461a-bdab-3705564defcd service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:08.943940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5baf-7e00-461a-bdab-3705564defcd service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:08.944214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5baf-7e00-461a-bdab-3705564defcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:08.945867 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a5e5baf-7e00-461a-bdab-3705564defcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:08.985993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:09.040847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e5baf-7e00-461a-bdab-3705564defcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:09.044233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e5baf-7e00-461a-bdab-3705564defcd service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:09.061122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 698/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:11.137800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b38a0-d515-4ac9-976b-6c3859723479 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:11.138327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b38a0-d515-4ac9-976b-6c3859723479 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:11.138327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b38a0-d515-4ac9-976b-6c3859723479 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:11.139758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 691/1389] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:11.145049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbe409-3e58-4826-b7da-05a00af4a06b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:11.145446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbe409-3e58-4826-b7da-05a00af4a06b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:11.145602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbe409-3e58-4826-b7da-05a00af4a06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:11.212347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cfcbe409-3e58-4826-b7da-05a00af4a06b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:11.213568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfcbe409-3e58-4826-b7da-05a00af4a06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:11.314636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 699/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:11.359673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b38a0-d515-4ac9-976b-6c3859723479 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:11.361364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-620b38a0-d515-4ac9-976b-6c3859723479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:11.394257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:11.521439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b38a0-d515-4ac9-976b-6c3859723479 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:11.632417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:11.644700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f69f0e-bf0b-4fc5-aaaf-35ea25077716 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:11.645316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f69f0e-bf0b-4fc5-aaaf-35ea25077716 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:11.645590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f69f0e-bf0b-4fc5-aaaf-35ea25077716 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:11.645855 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f69f0e-bf0b-4fc5-aaaf-35ea25077716 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:11.647492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d0f69f0e-bf0b-4fc5-aaaf-35ea25077716 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:11.717743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b14d32-5f0e-4caf-8d8b-8aaffd689a39 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:11.718032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b14d32-5f0e-4caf-8d8b-8aaffd689a39 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:11.718306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b14d32-5f0e-4caf-8d8b-8aaffd689a39 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:11.719726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 700/1391] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:51:11.725074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747076e-027e-4b97-b24c-cb3269b87d28 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:11.725467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747076e-027e-4b97-b24c-cb3269b87d28 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:11.725759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747076e-027e-4b97-b24c-cb3269b87d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:11.738110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f69f0e-bf0b-4fc5-aaaf-35ea25077716 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:11.739742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f69f0e-bf0b-4fc5-aaaf-35ea25077716 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:11.741435 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:11.758643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 692/1392] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:11.796942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b747076e-027e-4b97-b24c-cb3269b87d28 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:11.798171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b747076e-027e-4b97-b24c-cb3269b87d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:11.908535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 701/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:51:11.966382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00995a82-36cf-4852-85e1-21bd65eabf4c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:11.966703 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00995a82-36cf-4852-85e1-21bd65eabf4c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:11.966980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00995a82-36cf-4852-85e1-21bd65eabf4c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:11.968373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 693/1394] 10.222.0.20 () {54 vars in 966 bytes} [Sat Jul 30 04:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:11.973887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6694d39-ed99-41ef-9cdd-7f885e458de8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:11.974309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6694d39-ed99-41ef-9cdd-7f885e458de8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:11.974660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6694d39-ed99-41ef-9cdd-7f885e458de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:12.055596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b6694d39-ed99-41ef-9cdd-7f885e458de8 None None] MFA Rules not processed for user `548acf58bf9140ec9f1ba126e10deb45`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:12.056895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b6694d39-ed99-41ef-9cdd-7f885e458de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:12.094291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:12.137932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 702/1395] 10.222.0.20 () {58 vars in 1063 bytes} [Sat Jul 30 04:51:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:12.145716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00995a82-36cf-4852-85e1-21bd65eabf4c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:12.147282 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-00995a82-36cf-4852-85e1-21bd65eabf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:12.253217 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00995a82-36cf-4852-85e1-21bd65eabf4c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:12.267616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:12.329413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b392ce-ca7b-4547-af6c-c2804f1ef3dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '548acf58bf9140ec9f1ba126e10deb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:12.330027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b392ce-ca7b-4547-af6c-c2804f1ef3dd service glance] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:12.330301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b392ce-ca7b-4547-af6c-c2804f1ef3dd service glance] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:12.330612 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b392ce-ca7b-4547-af6c-c2804f1ef3dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:12.332232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-29b392ce-ca7b-4547-af6c-c2804f1ef3dd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:12.441626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b392ce-ca7b-4547-af6c-c2804f1ef3dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:12.443138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b392ce-ca7b-4547-af6c-c2804f1ef3dd service glance] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:12.462528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 694/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:51:13.198828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9391098-f1f1-48b2-bffe-3396106d0e63 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:13.199299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9391098-f1f1-48b2-bffe-3396106d0e63 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:13.199391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9391098-f1f1-48b2-bffe-3396106d0e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:13.269475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e9391098-f1f1-48b2-bffe-3396106d0e63 None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:13.270714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e9391098-f1f1-48b2-bffe-3396106d0e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:13.360251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 703/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 04:51:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:13.404186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671028-3ece-4cf9-ae5a-4bdfa01896fd None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:13.404681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671028-3ece-4cf9-ae5a-4bdfa01896fd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:13.404791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671028-3ece-4cf9-ae5a-4bdfa01896fd None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:13.406096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 695/1398] 10.222.0.20 () {54 vars in 943 bytes} [Sat Jul 30 04:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:13.412697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413d471-391e-4286-8e2f-68d1c906c82c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:13.412987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413d471-391e-4286-8e2f-68d1c906c82c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:13.413264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413d471-391e-4286-8e2f-68d1c906c82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:13.473769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:13.493841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4413d471-391e-4286-8e2f-68d1c906c82c None None] MFA Rules not processed for user `54bda2af62ee4ee39a9a350b7ce6430e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:13.495064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4413d471-391e-4286-8e2f-68d1c906c82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:13.600752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 704/1399] 10.222.0.20 () {58 vars in 1040 bytes} [Sat Jul 30 04:51:13 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:51:13.610306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d671028-3ece-4cf9-ae5a-4bdfa01896fd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:13.621678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1d671028-3ece-4cf9-ae5a-4bdfa01896fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:13.667296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:13.764864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d671028-3ece-4cf9-ae5a-4bdfa01896fd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:13.837596 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409270f-fb4d-4a2f-9f93-fe96682fd282 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:13.837933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409270f-fb4d-4a2f-9f93-fe96682fd282 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:13.838216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409270f-fb4d-4a2f-9f93-fe96682fd282 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:13.839760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 705/1400] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:51:13.845310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224724-13d8-4715-b937-613b3f3ab803 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:13.845793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224724-13d8-4715-b937-613b3f3ab803 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:13.846065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224724-13d8-4715-b937-613b3f3ab803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:13.878505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:13.879123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:13.879398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:13.879664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:13.881349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:13.912372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:13.917097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cb224724-13d8-4715-b937-613b3f3ab803 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:13.918297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb224724-13d8-4715-b937-613b3f3ab803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:13.968865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:13.970492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:13.974245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:13.979676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 696/1401] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 04:51:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 04:51:14.025138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 706/1402] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:51:14.069045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:14.070417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:14.185997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e359dfc-29d6-4d37-bc76-2f95d4018cd5 service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:14.262052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:14.313667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b03515-852e-4ef3-ac3d-61945ed4dc76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:14.314300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b03515-852e-4ef3-ac3d-61945ed4dc76 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:14.314655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b03515-852e-4ef3-ac3d-61945ed4dc76 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:14.314991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b03515-852e-4ef3-ac3d-61945ed4dc76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:14.316684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-09b03515-852e-4ef3-ac3d-61945ed4dc76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:14.420184 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b03515-852e-4ef3-ac3d-61945ed4dc76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:14.421796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b03515-852e-4ef3-ac3d-61945ed4dc76 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:14.442985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 697/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:16.517856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf6c6f-eb5e-4ef3-86c0-9fb2670d9f53 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:16.518393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf6c6f-eb5e-4ef3-86c0-9fb2670d9f53 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:16.518393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf6c6f-eb5e-4ef3-86c0-9fb2670d9f53 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:16.520078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 707/1404] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:16.525482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380452dd-0fab-48f4-be8f-666019b9f898 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:16.525975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380452dd-0fab-48f4-be8f-666019b9f898 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:16.526079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380452dd-0fab-48f4-be8f-666019b9f898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:16.600481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-380452dd-0fab-48f4-be8f-666019b9f898 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:16.601864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-380452dd-0fab-48f4-be8f-666019b9f898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:16.643672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:16.716135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 698/1405] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:16.772467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf6c6f-eb5e-4ef3-86c0-9fb2670d9f53 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:16.776272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e3bf6c6f-eb5e-4ef3-86c0-9fb2670d9f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:17.108548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf6c6f-eb5e-4ef3-86c0-9fb2670d9f53 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:17.159105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:17.246362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b901477-a71d-4cba-ba11-6490ef7291bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:17.247020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b901477-a71d-4cba-ba11-6490ef7291bb service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:17.247303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b901477-a71d-4cba-ba11-6490ef7291bb service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:17.247579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b901477-a71d-4cba-ba11-6490ef7291bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:17.249345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6b901477-a71d-4cba-ba11-6490ef7291bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:17.352433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b901477-a71d-4cba-ba11-6490ef7291bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:17.354179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b901477-a71d-4cba-ba11-6490ef7291bb service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:17.374270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 708/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:19.467821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45318-e3a7-4ba2-a32d-fc4c8c0c8160 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:19.468407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45318-e3a7-4ba2-a32d-fc4c8c0c8160 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:19.468407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45318-e3a7-4ba2-a32d-fc4c8c0c8160 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:19.474077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 699/1407] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:51:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:19.476452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb00c78-b29a-461b-ad9e-1dba8c59fc43 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:19.477335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb00c78-b29a-461b-ad9e-1dba8c59fc43 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:19.477850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb00c78-b29a-461b-ad9e-1dba8c59fc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:19.556612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:19.567516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8bb00c78-b29a-461b-ad9e-1dba8c59fc43 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:19.568754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8bb00c78-b29a-461b-ad9e-1dba8c59fc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:19.689599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 709/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:51:19.736370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45318-e3a7-4ba2-a32d-fc4c8c0c8160 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:19.738064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0dc45318-e3a7-4ba2-a32d-fc4c8c0c8160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:19.901971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45318-e3a7-4ba2-a32d-fc4c8c0c8160 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:19.924376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:20.040568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d0711-790c-4b68-8107-be8e24ccbe43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:20.041189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d0711-790c-4b68-8107-be8e24ccbe43 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:20.041481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d0711-790c-4b68-8107-be8e24ccbe43 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:20.041739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d0711-790c-4b68-8107-be8e24ccbe43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:20.044208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9d0d0711-790c-4b68-8107-be8e24ccbe43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:20.148442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d0711-790c-4b68-8107-be8e24ccbe43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:20.150261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d0711-790c-4b68-8107-be8e24ccbe43 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:20.172711 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 700/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:29.687608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=44 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:30.170176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:36.459991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28b2f3-1415-4012-84e9-bd6225e0db4c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:36.460540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28b2f3-1415-4012-84e9-bd6225e0db4c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:36.460540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28b2f3-1415-4012-84e9-bd6225e0db4c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:36.461908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 710/1410] 10.222.0.20 () {54 vars in 918 bytes} [Sat Jul 30 04:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:36.467047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d436e-7a0c-48f2-81f6-949b692eaea4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:36.467455 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d436e-7a0c-48f2-81f6-949b692eaea4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:36.467566 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d436e-7a0c-48f2-81f6-949b692eaea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:36.543230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2e1d436e-7a0c-48f2-81f6-949b692eaea4 None None] MFA Rules not processed for user `21b7c64636bd47e6b9a486b6cdaab4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:36.544460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2e1d436e-7a0c-48f2-81f6-949b692eaea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:36.650976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 701/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Sat Jul 30 04:51:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:36.665441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a2337-0739-498f-b2ef-f1f48d6f439b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:36.665726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a2337-0739-498f-b2ef-f1f48d6f439b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:36.666002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a2337-0739-498f-b2ef-f1f48d6f439b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:36.667384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 711/1412] 10.222.0.20 () {54 vars in 918 bytes} [Sat Jul 30 04:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:36.671997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929513-7d2c-49ca-b5b7-6a0ef5fb2371 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:36.672284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929513-7d2c-49ca-b5b7-6a0ef5fb2371 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:36.672561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929513-7d2c-49ca-b5b7-6a0ef5fb2371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:36.744817 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:36.748185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d6929513-7d2c-49ca-b5b7-6a0ef5fb2371 None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:36.750033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d6929513-7d2c-49ca-b5b7-6a0ef5fb2371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:36.867057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 702/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Sat Jul 30 04:51:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:39.313847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492a005-7c1c-4fa7-a91c-18cd8ab27fe5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:39.314359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492a005-7c1c-4fa7-a91c-18cd8ab27fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:39.314359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492a005-7c1c-4fa7-a91c-18cd8ab27fe5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:39.315665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:39.320803 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e110c3-98bc-4600-a481-5a597ffe9816 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:39.321210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e110c3-98bc-4600-a481-5a597ffe9816 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:39.321325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e110c3-98bc-4600-a481-5a597ffe9816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:39.387624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-75e110c3-98bc-4600-a481-5a597ffe9816 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:39.388816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-75e110c3-98bc-4600-a481-5a597ffe9816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:39.445198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:39.487331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:39.529891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492a005-7c1c-4fa7-a91c-18cd8ab27fe5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:39.531510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0492a005-7c1c-4fa7-a91c-18cd8ab27fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:39.678064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492a005-7c1c-4fa7-a91c-18cd8ab27fe5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:39.750734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:39.792513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41affe-0f7d-4ef6-92f8-7e3c0782eac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:39.793130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41affe-0f7d-4ef6-92f8-7e3c0782eac9 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:39.793406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41affe-0f7d-4ef6-92f8-7e3c0782eac9 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:39.793699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41affe-0f7d-4ef6-92f8-7e3c0782eac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:39.795345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c41affe-0f7d-4ef6-92f8-7e3c0782eac9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:39.885032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41affe-0f7d-4ef6-92f8-7e3c0782eac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:39.886665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41affe-0f7d-4ef6-92f8-7e3c0782eac9 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:39.905062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 713/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:40.810536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9adaf-77a9-4412-8e85-b6ce36a61436 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:40.811105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9adaf-77a9-4412-8e85-b6ce36a61436 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:40.811105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9adaf-77a9-4412-8e85-b6ce36a61436 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:40.812446 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:40.817711 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55bccf-0363-402f-b9b5-588c17786e01 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:40.818110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55bccf-0363-402f-b9b5-588c17786e01 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:40.818256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55bccf-0363-402f-b9b5-588c17786e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:40.894859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6d55bccf-0363-402f-b9b5-588c17786e01 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:40.896067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d55bccf-0363-402f-b9b5-588c17786e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:40.923190 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:40.992449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:41.146323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9adaf-77a9-4412-8e85-b6ce36a61436 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:41.147792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d7a9adaf-77a9-4412-8e85-b6ce36a61436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:41.294268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9adaf-77a9-4412-8e85-b6ce36a61436 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:41.333233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:41.408745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b536f62-60d1-49c0-87e0-2de890d5436f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:41.409291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536f62-60d1-49c0-87e0-2de890d5436f service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:41.409571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536f62-60d1-49c0-87e0-2de890d5436f service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:41.409844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536f62-60d1-49c0-87e0-2de890d5436f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:41.411493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0b536f62-60d1-49c0-87e0-2de890d5436f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:41.500415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b536f62-60d1-49c0-87e0-2de890d5436f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:41.501996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b536f62-60d1-49c0-87e0-2de890d5436f service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:41.519871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 705/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:42.406485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e1135-e9e2-4927-9349-2756240d5abc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:42.407005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e1135-e9e2-4927-9349-2756240d5abc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:42.407005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e1135-e9e2-4927-9349-2756240d5abc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:42.408326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:42.413225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80a28a-0de4-4b0d-9a96-a2a2604878a2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:42.413650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80a28a-0de4-4b0d-9a96-a2a2604878a2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:42.413781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80a28a-0de4-4b0d-9a96-a2a2604878a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:42.470498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:42.493035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1a80a28a-0de4-4b0d-9a96-a2a2604878a2 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:42.494236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1a80a28a-0de4-4b0d-9a96-a2a2604878a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:42.588289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:42.741006 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e1135-e9e2-4927-9349-2756240d5abc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:42.742602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f93e1135-e9e2-4927-9349-2756240d5abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:42.889750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e1135-e9e2-4927-9349-2756240d5abc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:42.902044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:43.001944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e7e61-9d3a-4c41-8d7f-ee5672d75290 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:43.002508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e7e61-9d3a-4c41-8d7f-ee5672d75290 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:43.002793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e7e61-9d3a-4c41-8d7f-ee5672d75290 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:43.003067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e7e61-9d3a-4c41-8d7f-ee5672d75290 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:43.004949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b2e7e61-9d3a-4c41-8d7f-ee5672d75290 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:43.092762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e7e61-9d3a-4c41-8d7f-ee5672d75290 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:43.094407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e7e61-9d3a-4c41-8d7f-ee5672d75290 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:43.112781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 716/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:43.972206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d791df7-29a4-4efc-9177-ed0b259b61d4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:43.972773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d791df7-29a4-4efc-9177-ed0b259b61d4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:43.972773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d791df7-29a4-4efc-9177-ed0b259b61d4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:43.974101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:43.979882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde91453-bb5e-4a0d-a2d4-655fb8b3f915 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:43.980314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde91453-bb5e-4a0d-a2d4-655fb8b3f915 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:43.980438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde91453-bb5e-4a0d-a2d4-655fb8b3f915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:44.001035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:44.058376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bde91453-bb5e-4a0d-a2d4-655fb8b3f915 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:44.059605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bde91453-bb5e-4a0d-a2d4-655fb8b3f915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:44.154160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:44.310103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d791df7-29a4-4efc-9177-ed0b259b61d4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:44.311567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0d791df7-29a4-4efc-9177-ed0b259b61d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:44.421116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:44.457423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d791df7-29a4-4efc-9177-ed0b259b61d4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:44.567822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21875a09-2cbb-4620-bdd9-7a6675d69dc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:44.568356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21875a09-2cbb-4620-bdd9-7a6675d69dc4 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:44.568630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21875a09-2cbb-4620-bdd9-7a6675d69dc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:44.568895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21875a09-2cbb-4620-bdd9-7a6675d69dc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:44.570531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-21875a09-2cbb-4620-bdd9-7a6675d69dc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:44.645318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:44.660160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21875a09-2cbb-4620-bdd9-7a6675d69dc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:44.661747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21875a09-2cbb-4620-bdd9-7a6675d69dc4 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:44.679596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 708/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:45.551366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c134092-5c93-42b6-85b5-ea1abc3b266c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:45.551926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c134092-5c93-42b6-85b5-ea1abc3b266c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:45.551926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c134092-5c93-42b6-85b5-ea1abc3b266c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:45.553295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:45.559009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba22c12-a4c5-4b43-ae20-c79796c3740a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:45.559434 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba22c12-a4c5-4b43-ae20-c79796c3740a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:45.559585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba22c12-a4c5-4b43-ae20-c79796c3740a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:45.636124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8ba22c12-a4c5-4b43-ae20-c79796c3740a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:45.637333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8ba22c12-a4c5-4b43-ae20-c79796c3740a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:45.733571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:45.887305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c134092-5c93-42b6-85b5-ea1abc3b266c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:45.888756 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c134092-5c93-42b6-85b5-ea1abc3b266c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:45.977140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:46.039285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c134092-5c93-42b6-85b5-ea1abc3b266c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:46.153492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e5212-0e36-46a2-bdd4-9d426211b17a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:46.154036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e5212-0e36-46a2-bdd4-9d426211b17a service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:46.154321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e5212-0e36-46a2-bdd4-9d426211b17a service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:46.154625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e5212-0e36-46a2-bdd4-9d426211b17a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:46.156260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b4e5212-0e36-46a2-bdd4-9d426211b17a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:46.204647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:46.246118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e5212-0e36-46a2-bdd4-9d426211b17a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:46.247753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e5212-0e36-46a2-bdd4-9d426211b17a service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:46.265678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 719/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:47.069210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc0d7f-4fad-4daf-a46c-1860251f430b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:47.069718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc0d7f-4fad-4daf-a46c-1860251f430b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:47.069828 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc0d7f-4fad-4daf-a46c-1860251f430b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:47.071189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:47.076685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb07fa-4554-4766-9ad3-a9729030cb5b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:47.077114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb07fa-4554-4766-9ad3-a9729030cb5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:47.077265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb07fa-4554-4766-9ad3-a9729030cb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:47.152681 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-99eb07fa-4554-4766-9ad3-a9729030cb5b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:47.153892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99eb07fa-4554-4766-9ad3-a9729030cb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:47.251544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:47.403878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc0d7f-4fad-4daf-a46c-1860251f430b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:47.405412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ecfc0d7f-4fad-4daf-a46c-1860251f430b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:47.454636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:47.552279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc0d7f-4fad-4daf-a46c-1860251f430b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:47.664017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770f5e1-e693-4f7b-ae55-54c9ba7a864e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:47.664524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770f5e1-e693-4f7b-ae55-54c9ba7a864e service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:47.664802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770f5e1-e693-4f7b-ae55-54c9ba7a864e service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:47.665072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770f5e1-e693-4f7b-ae55-54c9ba7a864e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:47.666807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7770f5e1-e693-4f7b-ae55-54c9ba7a864e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:47.673795 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:47.757732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770f5e1-e693-4f7b-ae55-54c9ba7a864e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:47.759361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770f5e1-e693-4f7b-ae55-54c9ba7a864e service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:47.778393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 711/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:48.586589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4052ee-0a0e-4712-a1c0-744d97a7fb9a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:48.587178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4052ee-0a0e-4712-a1c0-744d97a7fb9a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:48.587178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4052ee-0a0e-4712-a1c0-744d97a7fb9a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:48.589004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:48.593939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46855c1a-18e1-499a-945e-efb7c6b3c68a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:48.594347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46855c1a-18e1-499a-945e-efb7c6b3c68a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:48.594506 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46855c1a-18e1-499a-945e-efb7c6b3c68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:48.671600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-46855c1a-18e1-499a-945e-efb7c6b3c68a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:48.672798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-46855c1a-18e1-499a-945e-efb7c6b3c68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:48.739727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:48.768647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:48.918547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4052ee-0a0e-4712-a1c0-744d97a7fb9a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:48.919977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c4052ee-0a0e-4712-a1c0-744d97a7fb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:48.938660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:49.068062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4052ee-0a0e-4712-a1c0-744d97a7fb9a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:49.155354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:49.181932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da71769-eb3d-4283-bac6-60465162e9d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:49.182589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da71769-eb3d-4283-bac6-60465162e9d0 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:49.182861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da71769-eb3d-4283-bac6-60465162e9d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:49.183131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da71769-eb3d-4283-bac6-60465162e9d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:49.184811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0da71769-eb3d-4283-bac6-60465162e9d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:49.274917 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da71769-eb3d-4283-bac6-60465162e9d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:49.277175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da71769-eb3d-4283-bac6-60465162e9d0 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:49.295126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 722/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:50.127421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd552-25ac-4329-97b3-9798c2572eba None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:50.127964 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd552-25ac-4329-97b3-9798c2572eba None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:50.127964 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd552-25ac-4329-97b3-9798c2572eba None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:50.129321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:50.134358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601a956-7589-43dc-93c1-3a5bb300f854 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:50.134818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601a956-7589-43dc-93c1-3a5bb300f854 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:50.134929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601a956-7589-43dc-93c1-3a5bb300f854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:50.211063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f601a956-7589-43dc-93c1-3a5bb300f854 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:50.212273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f601a956-7589-43dc-93c1-3a5bb300f854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:50.253719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:50.311278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:50.467600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dd552-25ac-4329-97b3-9798c2572eba None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:50.469309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-482dd552-25ac-4329-97b3-9798c2572eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:50.614577 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dd552-25ac-4329-97b3-9798c2572eba None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:50.668262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:50.726111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac2500-d5eb-4aee-9b02-c994765823e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:50.726853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac2500-d5eb-4aee-9b02-c994765823e6 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:50.727194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac2500-d5eb-4aee-9b02-c994765823e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:50.727432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac2500-d5eb-4aee-9b02-c994765823e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:50.729172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-39ac2500-d5eb-4aee-9b02-c994765823e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:50.820694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac2500-d5eb-4aee-9b02-c994765823e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:50.822298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac2500-d5eb-4aee-9b02-c994765823e6 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:50.840756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 714/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:51.644202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd822c-5fa2-48ce-ab19-42c6b1106f06 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:51.644746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd822c-5fa2-48ce-ab19-42c6b1106f06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:51.644746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd822c-5fa2-48ce-ab19-42c6b1106f06 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:51.646043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:51.651044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84986066-7f2a-4abc-8499-032b38aa6575 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:51.651467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84986066-7f2a-4abc-8499-032b38aa6575 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:51.651596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84986066-7f2a-4abc-8499-032b38aa6575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:51.729045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-84986066-7f2a-4abc-8499-032b38aa6575 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:51.730233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-84986066-7f2a-4abc-8499-032b38aa6575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:51.732319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:51.828697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:51.983555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd822c-5fa2-48ce-ab19-42c6b1106f06 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:51.985081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11fd822c-5fa2-48ce-ab19-42c6b1106f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:52.130732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd822c-5fa2-48ce-ab19-42c6b1106f06 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:52.158628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:52.243939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc54d6e-e547-41bc-b777-fe990daedef3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:52.244475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc54d6e-e547-41bc-b777-fe990daedef3 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:52.244751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc54d6e-e547-41bc-b777-fe990daedef3 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:52.245021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc54d6e-e547-41bc-b777-fe990daedef3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:52.246928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dbc54d6e-e547-41bc-b777-fe990daedef3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:52.335015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc54d6e-e547-41bc-b777-fe990daedef3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:52.336600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc54d6e-e547-41bc-b777-fe990daedef3 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:52.354146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 725/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:54.267708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbeccd-887e-44b9-8e0a-c8f8e330c1d3 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:54.268266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbeccd-887e-44b9-8e0a-c8f8e330c1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:54.268266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbeccd-887e-44b9-8e0a-c8f8e330c1d3 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:54.269632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:54.275239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a0b92-174f-404d-bbaf-707b87a6166d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:54.275661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a0b92-174f-404d-bbaf-707b87a6166d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:54.275764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a0b92-174f-404d-bbaf-707b87a6166d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:54.316419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:54.342782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-553a0b92-174f-404d-bbaf-707b87a6166d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:54.344000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-553a0b92-174f-404d-bbaf-707b87a6166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:54.440668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:54.478233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbeccd-887e-44b9-8e0a-c8f8e330c1d3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:54.479824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7acbeccd-887e-44b9-8e0a-c8f8e330c1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:54.582616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbeccd-887e-44b9-8e0a-c8f8e330c1d3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:54.614672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:54.656390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5c00c-d9aa-435a-a2a5-682d84e31fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:54.657005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5c00c-d9aa-435a-a2a5-682d84e31fde service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:54.657285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5c00c-d9aa-435a-a2a5-682d84e31fde service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:54.657557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5c00c-d9aa-435a-a2a5-682d84e31fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:54.659240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-50a5c00c-d9aa-435a-a2a5-682d84e31fde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:54.767541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5c00c-d9aa-435a-a2a5-682d84e31fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:54.768980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5c00c-d9aa-435a-a2a5-682d84e31fde service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:54.787592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 717/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:56.944884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4677a-8fec-4a1d-a238-c35501232e50 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:56.945427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4677a-8fec-4a1d-a238-c35501232e50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:56.945552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4677a-8fec-4a1d-a238-c35501232e50 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:56.946859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:56.952695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33488c-a7da-4eb3-9f66-6d8d4ce86283 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:56.953094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33488c-a7da-4eb3-9f66-6d8d4ce86283 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:56.953246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33488c-a7da-4eb3-9f66-6d8d4ce86283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:57.020048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8e33488c-a7da-4eb3-9f66-6d8d4ce86283 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:57.021266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8e33488c-a7da-4eb3-9f66-6d8d4ce86283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:57.034115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:57.118732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:51:57.158233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4677a-8fec-4a1d-a238-c35501232e50 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:51:57.159824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1bb4677a-8fec-4a1d-a238-c35501232e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:57.264955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4677a-8fec-4a1d-a238-c35501232e50 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:51:57.276975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:57.340652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cada71-4620-4c62-b208-023d0f1655d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:51:57.341267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cada71-4620-4c62-b208-023d0f1655d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:57.341550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cada71-4620-4c62-b208-023d0f1655d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:57.341895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cada71-4620-4c62-b208-023d0f1655d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:57.343706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f4cada71-4620-4c62-b208-023d0f1655d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:51:57.454128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cada71-4620-4c62-b208-023d0f1655d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:51:57.455592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cada71-4620-4c62-b208-023d0f1655d7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:51:57.473354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 728/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:51:59.885639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ba598-4410-4bb8-afc3-4df0edbd654b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:59.886482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ba598-4410-4bb8-afc3-4df0edbd654b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:59.886482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ba598-4410-4bb8-afc3-4df0edbd654b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:59.887807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:51:59.893585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59c09-ec4d-4495-8a15-d17321ca66ba None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:51:59.894025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59c09-ec4d-4495-8a15-d17321ca66ba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:51:59.894123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59c09-ec4d-4495-8a15-d17321ca66ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:51:59.943557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:51:59.961548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-70b59c09-ec4d-4495-8a15-d17321ca66ba None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:51:59.962789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-70b59c09-ec4d-4495-8a15-d17321ca66ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:00.061605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:00.101057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ba598-4410-4bb8-afc3-4df0edbd654b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:00.102632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d00ba598-4410-4bb8-afc3-4df0edbd654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:00.207807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ba598-4410-4bb8-afc3-4df0edbd654b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:00.248711 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:00.284399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1040d9-7a0d-4cc4-908c-0e87d4cc89d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:00.285036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1040d9-7a0d-4cc4-908c-0e87d4cc89d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:00.285308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1040d9-7a0d-4cc4-908c-0e87d4cc89d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:00.285576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1040d9-7a0d-4cc4-908c-0e87d4cc89d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:00.287295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9c1040d9-7a0d-4cc4-908c-0e87d4cc89d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:00.394302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1040d9-7a0d-4cc4-908c-0e87d4cc89d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:00.395779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1040d9-7a0d-4cc4-908c-0e87d4cc89d9 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:00.413583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 720/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:02.549064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf944f0-1590-423e-ba7e-4b49cf3c6ee0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:02.549652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf944f0-1590-423e-ba7e-4b49cf3c6ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:02.549652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf944f0-1590-423e-ba7e-4b49cf3c6ee0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:02.551038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:02.556291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7447a-9adc-4748-bae1-91da4793f128 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:02.556726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7447a-9adc-4748-bae1-91da4793f128 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:02.556842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f7447a-9adc-4748-bae1-91da4793f128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:02.622236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f3f7447a-9adc-4748-bae1-91da4793f128 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:02.623456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f3f7447a-9adc-4748-bae1-91da4793f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:02.644883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:02.722099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:02.760441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf944f0-1590-423e-ba7e-4b49cf3c6ee0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:02.762241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6cf944f0-1590-423e-ba7e-4b49cf3c6ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:02.867555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf944f0-1590-423e-ba7e-4b49cf3c6ee0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:02.887766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:02.941362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620bc2d-dba5-43ee-b516-b73148c9ce39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:02.941972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620bc2d-dba5-43ee-b516-b73148c9ce39 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:02.942249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620bc2d-dba5-43ee-b516-b73148c9ce39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:02.942549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620bc2d-dba5-43ee-b516-b73148c9ce39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:02.944161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2620bc2d-dba5-43ee-b516-b73148c9ce39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:03.053254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620bc2d-dba5-43ee-b516-b73148c9ce39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:03.054727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620bc2d-dba5-43ee-b516-b73148c9ce39 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:03.073508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 731/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:05.116487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78da1b6-1373-46f1-aea2-fcfdb1b0e6ba None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:05.117031 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78da1b6-1373-46f1-aea2-fcfdb1b0e6ba None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:05.117127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78da1b6-1373-46f1-aea2-fcfdb1b0e6ba None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:05.118457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:05.123743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab336-ab61-4548-a792-8d0b74580db4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:05.124155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab336-ab61-4548-a792-8d0b74580db4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:05.124291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab336-ab61-4548-a792-8d0b74580db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:05.189184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:05.192118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f5cab336-ab61-4548-a792-8d0b74580db4 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:05.194105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5cab336-ab61-4548-a792-8d0b74580db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:05.290311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:05.329277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78da1b6-1373-46f1-aea2-fcfdb1b0e6ba None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:05.330865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c78da1b6-1373-46f1-aea2-fcfdb1b0e6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:05.434390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78da1b6-1373-46f1-aea2-fcfdb1b0e6ba None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:05.483172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:05.509281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80822de-7fe5-42f4-b2b1-15f87739b8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:05.510130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80822de-7fe5-42f4-b2b1-15f87739b8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:05.510452 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80822de-7fe5-42f4-b2b1-15f87739b8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:05.510735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80822de-7fe5-42f4-b2b1-15f87739b8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:05.512373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e80822de-7fe5-42f4-b2b1-15f87739b8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:05.621760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80822de-7fe5-42f4-b2b1-15f87739b8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:05.623166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80822de-7fe5-42f4-b2b1-15f87739b8c6 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:05.642396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 723/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:07.712329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cca41d-bc5e-49e6-ac29-87ef45b79ab4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:07.712878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cca41d-bc5e-49e6-ac29-87ef45b79ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:07.712878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cca41d-bc5e-49e6-ac29-87ef45b79ab4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:07.714327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:52:07.719561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221030a-7af8-4bc9-a65f-d0b0ca4134e5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:07.720056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221030a-7af8-4bc9-a65f-d0b0ca4134e5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:07.720242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221030a-7af8-4bc9-a65f-d0b0ca4134e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:07.788752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2221030a-7af8-4bc9-a65f-d0b0ca4134e5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:07.789951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2221030a-7af8-4bc9-a65f-d0b0ca4134e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:07.817975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:07.885535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:07.926459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cca41d-bc5e-49e6-ac29-87ef45b79ab4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:07.928054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66cca41d-bc5e-49e6-ac29-87ef45b79ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:08.035337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cca41d-bc5e-49e6-ac29-87ef45b79ab4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:08.063651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:08.109897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3665a-5cf1-4d8d-92d6-ff8f3d3c776e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:08.110544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3665a-5cf1-4d8d-92d6-ff8f3d3c776e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:08.110929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3665a-5cf1-4d8d-92d6-ff8f3d3c776e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:08.111237 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3665a-5cf1-4d8d-92d6-ff8f3d3c776e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:08.112968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7ad3665a-5cf1-4d8d-92d6-ff8f3d3c776e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:08.224585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3665a-5cf1-4d8d-92d6-ff8f3d3c776e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:08.225995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3665a-5cf1-4d8d-92d6-ff8f3d3c776e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:08.244757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 734/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:10.332829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ad896-d94a-4389-b10f-236be9c3e89b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:10.333379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ad896-d94a-4389-b10f-236be9c3e89b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:10.333379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ad896-d94a-4389-b10f-236be9c3e89b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:10.334701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:10.339992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e5fc8-eb1a-4e50-be06-1ff6a8baed4e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:10.340375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e5fc8-eb1a-4e50-be06-1ff6a8baed4e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:10.340559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e5fc8-eb1a-4e50-be06-1ff6a8baed4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:10.407663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-079e5fc8-eb1a-4e50-be06-1ff6a8baed4e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:10.408978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-079e5fc8-eb1a-4e50-be06-1ff6a8baed4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:10.417185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:10.506144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:10.544755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ad896-d94a-4389-b10f-236be9c3e89b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:10.546294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-027ad896-d94a-4389-b10f-236be9c3e89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:10.653605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ad896-d94a-4389-b10f-236be9c3e89b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:10.710579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:10.730876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d807f-8ad8-482c-8d8e-362c0ba168a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:10.731500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d807f-8ad8-482c-8d8e-362c0ba168a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:10.731779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d807f-8ad8-482c-8d8e-362c0ba168a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:10.732048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d807f-8ad8-482c-8d8e-362c0ba168a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:10.733678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8e8d807f-8ad8-482c-8d8e-362c0ba168a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:10.841644 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d807f-8ad8-482c-8d8e-362c0ba168a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:10.843117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d807f-8ad8-482c-8d8e-362c0ba168a0 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:10.861684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 726/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:12.948540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d21d2-01c6-458b-9817-400a9bc2174c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:12.949131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d21d2-01c6-458b-9817-400a9bc2174c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:12.949131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d21d2-01c6-458b-9817-400a9bc2174c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:12.950491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:52:12.955854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef4800-04d7-4eba-afc3-9a7342242364 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:12.956439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef4800-04d7-4eba-afc3-9a7342242364 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:12.956439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef4800-04d7-4eba-afc3-9a7342242364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:13.023295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-50ef4800-04d7-4eba-afc3-9a7342242364 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:13.024510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-50ef4800-04d7-4eba-afc3-9a7342242364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:13.062750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:13.120452 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:13.158873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d21d2-01c6-458b-9817-400a9bc2174c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:13.160447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc8d21d2-01c6-458b-9817-400a9bc2174c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:13.265311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d21d2-01c6-458b-9817-400a9bc2174c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:13.304219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:13.340845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adafbdd-013a-4a33-a7de-4ee9f4b89cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:13.341458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adafbdd-013a-4a33-a7de-4ee9f4b89cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:13.341737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adafbdd-013a-4a33-a7de-4ee9f4b89cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:13.342010 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adafbdd-013a-4a33-a7de-4ee9f4b89cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:13.343656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9adafbdd-013a-4a33-a7de-4ee9f4b89cc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:13.454395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adafbdd-013a-4a33-a7de-4ee9f4b89cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:13.455851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adafbdd-013a-4a33-a7de-4ee9f4b89cc5 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:13.474340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 737/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:15.500026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97125-7efd-4aac-81fd-26072425f909 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:15.500602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97125-7efd-4aac-81fd-26072425f909 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:15.500602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97125-7efd-4aac-81fd-26072425f909 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:15.501961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:15.507250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969e4b-bf6f-434e-91a9-0848b6c3e722 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:15.507690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969e4b-bf6f-434e-91a9-0848b6c3e722 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:15.507797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38969e4b-bf6f-434e-91a9-0848b6c3e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:15.574936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-38969e4b-bf6f-434e-91a9-0848b6c3e722 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:15.576138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38969e4b-bf6f-434e-91a9-0848b6c3e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:15.595480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:15.675992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:15.713309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97125-7efd-4aac-81fd-26072425f909 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:15.714937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2bd97125-7efd-4aac-81fd-26072425f909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:15.817715 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97125-7efd-4aac-81fd-26072425f909 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:15.884385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:15.891793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a2c38-99a2-4d5c-b2f8-abb84dfd9742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:15.892412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a2c38-99a2-4d5c-b2f8-abb84dfd9742 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:15.892692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a2c38-99a2-4d5c-b2f8-abb84dfd9742 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:15.892965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a2c38-99a2-4d5c-b2f8-abb84dfd9742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:15.895214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-994a2c38-99a2-4d5c-b2f8-abb84dfd9742 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:16.127679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a2c38-99a2-4d5c-b2f8-abb84dfd9742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:16.129250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a2c38-99a2-4d5c-b2f8-abb84dfd9742 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:16.147967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 729/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:18.189224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273a9a1-3ea9-4c78-8d13-7913beb020ae None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:18.189777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273a9a1-3ea9-4c78-8d13-7913beb020ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:18.189777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273a9a1-3ea9-4c78-8d13-7913beb020ae None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:18.191158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:18.196717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db18d-1de0-4f7d-be8f-fbf04bd6ebe8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:18.197174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db18d-1de0-4f7d-be8f-fbf04bd6ebe8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:18.197284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db18d-1de0-4f7d-be8f-fbf04bd6ebe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:18.261386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c53db18d-1de0-4f7d-be8f-fbf04bd6ebe8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:18.262608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c53db18d-1de0-4f7d-be8f-fbf04bd6ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:18.311127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:18.359764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:18.398625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273a9a1-3ea9-4c78-8d13-7913beb020ae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:18.400189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a273a9a1-3ea9-4c78-8d13-7913beb020ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:18.503859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273a9a1-3ea9-4c78-8d13-7913beb020ae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:18.553672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:18.579171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f6957-094f-4bc7-8220-2bbb7f0cba1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:18.579721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6957-094f-4bc7-8220-2bbb7f0cba1c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:18.580015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6957-094f-4bc7-8220-2bbb7f0cba1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:18.580270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6957-094f-4bc7-8220-2bbb7f0cba1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:18.581874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-332f6957-094f-4bc7-8220-2bbb7f0cba1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:18.689471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f6957-094f-4bc7-8220-2bbb7f0cba1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:18.691008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f6957-094f-4bc7-8220-2bbb7f0cba1c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:18.709164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 740/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:28.358047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=22 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:28.706714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=66 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:38.416769 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51142424-6d8b-4c99-a952-a29b049db329 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:38.417305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51142424-6d8b-4c99-a952-a29b049db329 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:38.417305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51142424-6d8b-4c99-a952-a29b049db329 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:38.418636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 731/1471] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:52:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:38.423840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1da02-f5fe-42d7-bb8e-466767a37836 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:38.424323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1da02-f5fe-42d7-bb8e-466767a37836 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:38.424435 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1da02-f5fe-42d7-bb8e-466767a37836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:38.492102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f3f1da02-f5fe-42d7-bb8e-466767a37836 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:38.493314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f3f1da02-f5fe-42d7-bb8e-466767a37836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:38.590236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 741/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:38.675454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02b77-eb2a-45aa-a658-a0f68f7e2476 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:38.675741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02b77-eb2a-45aa-a658-a0f68f7e2476 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:38.676019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02b77-eb2a-45aa-a658-a0f68f7e2476 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:38.677361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 732/1473] 10.222.0.20 () {54 vars in 966 bytes} [Sat Jul 30 04:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:38.682871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee712d8-d811-4a23-b646-0968289378ad None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:38.683163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee712d8-d811-4a23-b646-0968289378ad None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:38.683438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee712d8-d811-4a23-b646-0968289378ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:38.760491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6ee712d8-d811-4a23-b646-0968289378ad None None] MFA Rules not processed for user `548acf58bf9140ec9f1ba126e10deb45`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:38.761722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ee712d8-d811-4a23-b646-0968289378ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:38.782383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:38.839219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 742/1474] 10.222.0.20 () {58 vars in 1063 bytes} [Sat Jul 30 04:52:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:38.846721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef02b77-eb2a-45aa-a658-a0f68f7e2476 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:38.848233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2ef02b77-eb2a-45aa-a658-a0f68f7e2476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:38.950989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef02b77-eb2a-45aa-a658-a0f68f7e2476 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:39.023515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5298f-a36b-41b2-bacf-48646bbdb359 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '548acf58bf9140ec9f1ba126e10deb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:39.024136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5298f-a36b-41b2-bacf-48646bbdb359 service glance] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:39.024411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5298f-a36b-41b2-bacf-48646bbdb359 service glance] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:39.024680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5298f-a36b-41b2-bacf-48646bbdb359 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:39.027169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-99f5298f-a36b-41b2-bacf-48646bbdb359 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:39.080631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:39.135682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5298f-a36b-41b2-bacf-48646bbdb359 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:39.137101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5298f-a36b-41b2-bacf-48646bbdb359 service glance] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:39.155781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 733/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:41.252626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679dfac-6378-46b3-a885-0ff202b1b36b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:41.253186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679dfac-6378-46b3-a885-0ff202b1b36b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:41.253186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679dfac-6378-46b3-a885-0ff202b1b36b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:41.254562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 743/1476] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:41.260023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a684917-3013-42a1-a959-89660b084798 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:41.261047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a684917-3013-42a1-a959-89660b084798 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:41.261240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a684917-3013-42a1-a959-89660b084798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:41.325959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5a684917-3013-42a1-a959-89660b084798 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:41.327133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5a684917-3013-42a1-a959-89660b084798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:41.426563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 734/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:52:41.451027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679dfac-6378-46b3-a885-0ff202b1b36b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:41.452585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3679dfac-6378-46b3-a885-0ff202b1b36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:41.575841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679dfac-6378-46b3-a885-0ff202b1b36b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:41.624743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:41.671350 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e3fb5-24dc-41e3-9613-da81cdceb60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:41.671963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3fb5-24dc-41e3-9613-da81cdceb60e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:41.672860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3fb5-24dc-41e3-9613-da81cdceb60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:41.673140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e3fb5-24dc-41e3-9613-da81cdceb60e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:41.673828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3fb5-24dc-41e3-9613-da81cdceb60e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:41.675566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-b07e3fb5-24dc-41e3-9613-da81cdceb60e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:52:41.675566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 30 04:52:41.675566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: Jul 30 04:52:41.676691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e3fb5-24dc-41e3-9613-da81cdceb60e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:41.695920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 744/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Jul 30 04:52:41 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:52:43.712589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67472d1e-191c-4cf2-92bd-da0f09f709f8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:43.713124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67472d1e-191c-4cf2-92bd-da0f09f709f8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:43.713124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67472d1e-191c-4cf2-92bd-da0f09f709f8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:43.714664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 735/1479] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:43.719944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5e1ad-7d85-4ba1-bfa8-c867358c55e1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:43.720404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5e1ad-7d85-4ba1-bfa8-c867358c55e1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:43.720506 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5e1ad-7d85-4ba1-bfa8-c867358c55e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:43.787150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9ff5e1ad-7d85-4ba1-bfa8-c867358c55e1 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:43.788351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ff5e1ad-7d85-4ba1-bfa8-c867358c55e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:43.888075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 745/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:52:43.930640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67472d1e-191c-4cf2-92bd-da0f09f709f8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:43.932356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-67472d1e-191c-4cf2-92bd-da0f09f709f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:43.942095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:44.075309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67472d1e-191c-4cf2-92bd-da0f09f709f8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:44.159212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:44.191920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355796f-cb18-41b2-a736-dcdc2045c824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:44.192538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355796f-cb18-41b2-a736-dcdc2045c824 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:44.192817 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355796f-cb18-41b2-a736-dcdc2045c824 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:44.193087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355796f-cb18-41b2-a736-dcdc2045c824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:44.194746 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a355796f-cb18-41b2-a736-dcdc2045c824 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:44.284706 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355796f-cb18-41b2-a736-dcdc2045c824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:44.286209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355796f-cb18-41b2-a736-dcdc2045c824 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:44.304011 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 736/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:46.490983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8846d12-8eb0-4a6f-bb75-177b62b5f2d7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:46.491529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8846d12-8eb0-4a6f-bb75-177b62b5f2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:46.491639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8846d12-8eb0-4a6f-bb75-177b62b5f2d7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:46.492938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 746/1482] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:46.498232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f403d89-cd3d-49b4-a7fb-b8bcabc04282 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:46.498727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f403d89-cd3d-49b4-a7fb-b8bcabc04282 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:46.498828 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f403d89-cd3d-49b4-a7fb-b8bcabc04282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:46.564583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2f403d89-cd3d-49b4-a7fb-b8bcabc04282 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:46.565801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2f403d89-cd3d-49b4-a7fb-b8bcabc04282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:46.611815 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:46.662170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 737/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:46.699256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8846d12-8eb0-4a6f-bb75-177b62b5f2d7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:46.700902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8846d12-8eb0-4a6f-bb75-177b62b5f2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:46.724714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:46.805702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8846d12-8eb0-4a6f-bb75-177b62b5f2d7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:46.878917 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568404d-3287-4820-b9ac-cf47365f5e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:46.879535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568404d-3287-4820-b9ac-cf47365f5e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:46.879818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568404d-3287-4820-b9ac-cf47365f5e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:46.880096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568404d-3287-4820-b9ac-cf47365f5e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:46.881731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1568404d-3287-4820-b9ac-cf47365f5e9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:46.956202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:46.991512 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568404d-3287-4820-b9ac-cf47365f5e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:46.992953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568404d-3287-4820-b9ac-cf47365f5e9b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:47.011924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 747/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:49.107606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49163a2-f1cc-4289-8744-bd3952483ef0 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:49.108337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49163a2-f1cc-4289-8744-bd3952483ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:49.108337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49163a2-f1cc-4289-8744-bd3952483ef0 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:49.109504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 738/1485] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:49.114821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccf02a-8f08-4e81-8860-9a6483d2ff94 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:49.115227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccf02a-8f08-4e81-8860-9a6483d2ff94 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:49.115388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccf02a-8f08-4e81-8860-9a6483d2ff94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:49.181809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fdccf02a-8f08-4e81-8860-9a6483d2ff94 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:49.183068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fdccf02a-8f08-4e81-8860-9a6483d2ff94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:49.280210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 748/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:49.319139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49163a2-f1cc-4289-8744-bd3952483ef0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:49.320688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d49163a2-f1cc-4289-8744-bd3952483ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:49.425073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49163a2-f1cc-4289-8744-bd3952483ef0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:49.502054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:49.503826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6558a21-35f7-4792-981c-ccce8e4be9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:49.504561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558a21-35f7-4792-981c-ccce8e4be9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:49.504877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558a21-35f7-4792-981c-ccce8e4be9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:49.505370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558a21-35f7-4792-981c-ccce8e4be9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:49.507482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a6558a21-35f7-4792-981c-ccce8e4be9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:49.618390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6558a21-35f7-4792-981c-ccce8e4be9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:49.619783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6558a21-35f7-4792-981c-ccce8e4be9e4 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:49.639207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 739/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:51.807739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f546f50-ac1c-4440-a40e-762ae2f756e8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:51.808305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f546f50-ac1c-4440-a40e-762ae2f756e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:51.808305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f546f50-ac1c-4440-a40e-762ae2f756e8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:51.810310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 749/1488] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:51.815500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c71656-2daf-4b5b-be39-a0e598823bac None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:51.816018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c71656-2daf-4b5b-be39-a0e598823bac None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:51.816018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c71656-2daf-4b5b-be39-a0e598823bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:51.881672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c4c71656-2daf-4b5b-be39-a0e598823bac None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:51.882982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c4c71656-2daf-4b5b-be39-a0e598823bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:51.936239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:51.980261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 740/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:52.021828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f546f50-ac1c-4440-a40e-762ae2f756e8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:52.023482 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f546f50-ac1c-4440-a40e-762ae2f756e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:52.061456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:52.130855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f546f50-ac1c-4440-a40e-762ae2f756e8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:52.204373 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226f1ea-31c7-4a4f-b8a8-596f607b815c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:52.204989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226f1ea-31c7-4a4f-b8a8-596f607b815c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:52.205264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226f1ea-31c7-4a4f-b8a8-596f607b815c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:52.205534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226f1ea-31c7-4a4f-b8a8-596f607b815c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:52.207211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4226f1ea-31c7-4a4f-b8a8-596f607b815c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:52.286873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:52.317083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226f1ea-31c7-4a4f-b8a8-596f607b815c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:52.318625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226f1ea-31c7-4a4f-b8a8-596f607b815c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:52.336549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 750/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:52:54.625484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d0af3-a13e-4fa3-b017-3214a12a0b6f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:54.626058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d0af3-a13e-4fa3-b017-3214a12a0b6f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:54.626058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d0af3-a13e-4fa3-b017-3214a12a0b6f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:54.627404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 741/1491] 10.222.0.20 () {54 vars in 926 bytes} [Sat Jul 30 04:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:52:54.632502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184f19b-fdcf-477f-a7c8-f4faa81262fb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:54.632939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184f19b-fdcf-477f-a7c8-f4faa81262fb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:54.633091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184f19b-fdcf-477f-a7c8-f4faa81262fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:54.702044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9184f19b-fdcf-477f-a7c8-f4faa81262fb None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:52:54.703297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9184f19b-fdcf-477f-a7c8-f4faa81262fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:54.802785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 751/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Sat Jul 30 04:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:52:54.842596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d0af3-a13e-4fa3-b017-3214a12a0b6f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:52:54.844148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ea7d0af3-a13e-4fa3-b017-3214a12a0b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:54.947238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d0af3-a13e-4fa3-b017-3214a12a0b6f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:52:55.020663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59a893-6882-453c-91e2-4d641a15a8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:52:55.021284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59a893-6882-453c-91e2-4d641a15a8ee service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:52:55.021564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59a893-6882-453c-91e2-4d641a15a8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:52:55.021842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59a893-6882-453c-91e2-4d641a15a8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:52:55.023511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-da59a893-6882-453c-91e2-4d641a15a8ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:52:55.033336 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:52:55.132975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59a893-6882-453c-91e2-4d641a15a8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:52:55.134405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59a893-6882-453c-91e2-4d641a15a8ee service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:52:55.151865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 742/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:53:04.800927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:53:05.149657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=50 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:28.689593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99b338-489a-489c-a049-fab8731a6558 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:28.690193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99b338-489a-489c-a049-fab8731a6558 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:28.690193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99b338-489a-489c-a049-fab8731a6558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:28.773590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eb99b338-489a-489c-a049-fab8731a6558 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:54:28.774924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eb99b338-489a-489c-a049-fab8731a6558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:28.876780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 752/1494] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:54:28.885560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abc1ea-eb6f-4191-9732-00c716f60c0d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:28.886052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abc1ea-eb6f-4191-9732-00c716f60c0d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:28.886240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abc1ea-eb6f-4191-9732-00c716f60c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:28.961265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c1abc1ea-eb6f-4191-9732-00c716f60c0d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:54:28.962535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c1abc1ea-eb6f-4191-9732-00c716f60c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:29.058065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 743/1495] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:54:29.062753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99b338-489a-489c-a049-fab8731a6558 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:29.063979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eb99b338-489a-489c-a049-fab8731a6558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:29.075963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99b338-489a-489c-a049-fab8731a6558 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:29.136964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:29.171485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:29.172098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:29.172375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:29.172643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:29.173329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:54:29.173623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:29.174654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:54:29.174654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 30 04:54:29.174654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: Jul 30 04:54:29.175534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:29.187383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 753/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:54:29.195805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abc1ea-eb6f-4191-9732-00c716f60c0d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:29.204933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abc1ea-eb6f-4191-9732-00c716f60c0d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:29.265352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:29.299614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c547969-c24c-4591-b290-609175841837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:29.300226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c547969-c24c-4591-b290-609175841837 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:29.300502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c547969-c24c-4591-b290-609175841837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:29.300770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c547969-c24c-4591-b290-609175841837 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:29.301430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c547969-c24c-4591-b290-609175841837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:29.303197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-3c547969-c24c-4591-b290-609175841837 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:54:29.303197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 30 04:54:29.303197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: Jul 30 04:54:29.304384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c547969-c24c-4591-b290-609175841837 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:29.360175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 744/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:54:29 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:54:29.364564 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:29.366096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:29.399336 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7782d-708d-4985-804c-afc51285d054 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:29.498708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:29.499329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:29.499605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:29.499874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:29.500535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:29.501543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:54:29.501543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 30 04:54:29.501543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: Jul 30 04:54:29.502761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:29.544752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 754/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:54:29 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:54:29.548945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c547969-c24c-4591-b290-609175841837 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:29.550520 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3c547969-c24c-4591-b290-609175841837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:29.612451 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c547969-c24c-4591-b290-609175841837 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:29.697526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:29.708909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:29.709522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:29.709795 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:29.710059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:29.710778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:29.712373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:54:29.712373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 30 04:54:29.712373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: Jul 30 04:54:29.713236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:29.725629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 745/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:54:29.730351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:29.731903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:29.744020 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049ad31-c256-4eb6-a6af-1ac393bf035f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:29.769214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:29.845210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6481940-502e-4cdd-83d1-068975b21971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:29.845747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6481940-502e-4cdd-83d1-068975b21971 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:29.846027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6481940-502e-4cdd-83d1-068975b21971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:29.846300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6481940-502e-4cdd-83d1-068975b21971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:29.847027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6481940-502e-4cdd-83d1-068975b21971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:29.848035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-d6481940-502e-4cdd-83d1-068975b21971 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:54:29.848035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 30 04:54:29.848035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: Jul 30 04:54:29.848923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6481940-502e-4cdd-83d1-068975b21971 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:29.861745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 755/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:54:29.866077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:29.867630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:29.901202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d67a03-9d32-4e28-b69a-46d645fb5526 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:29.994351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583dec8-bf40-46e1-9c69-7b3c67f27d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:29.995040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583dec8-bf40-46e1-9c69-7b3c67f27d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:29.995317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583dec8-bf40-46e1-9c69-7b3c67f27d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:29.995587 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583dec8-bf40-46e1-9c69-7b3c67f27d03 admin admin] PATH_INFO: `/v3/projects/d34c8aa4cc754824b9e9f116dbd66819/users/c6ac3002dc024090a91c29c571f0ac35/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:30.017945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583dec8-bf40-46e1-9c69-7b3c67f27d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34c8aa4cc754824b9e9f116dbd66819, user_id=c6ac3002dc024090a91c29c571f0ac35, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:30.019286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-2583dec8-bf40-46e1-9c69-7b3c67f27d03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:54:30.019286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 30 04:54:30.019286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: Jul 30 04:54:30.020964 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583dec8-bf40-46e1-9c69-7b3c67f27d03 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:30.046747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 746/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:54:29 2022] PUT /identity/v3/projects/d34c8aa4cc754824b9e9f116dbd66819/users/c6ac3002dc024090a91c29c571f0ac35/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:54:30.059769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b701590-8e24-4a0c-86f0-d3348ded16f4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:30.060065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b701590-8e24-4a0c-86f0-d3348ded16f4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:30.060346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b701590-8e24-4a0c-86f0-d3348ded16f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:30.117033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5b701590-8e24-4a0c-86f0-d3348ded16f4 None None] MFA Rules not processed for user `c6ac3002dc024090a91c29c571f0ac35`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:54:30.118872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5b701590-8e24-4a0c-86f0-d3348ded16f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:30.195213 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 756/1502] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:54:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:54:30.208335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624497f-0002-4b5c-b627-c635034d4629 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:30.208640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624497f-0002-4b5c-b627-c635034d4629 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:30.208895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624497f-0002-4b5c-b627-c635034d4629 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:30.210245 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 747/1503] 10.222.0.20 () {54 vars in 964 bytes} [Sat Jul 30 04:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:54:30.215746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c648be4-f9bc-4fda-900f-05ca2b134d09 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:30.216033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c648be4-f9bc-4fda-900f-05ca2b134d09 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:30.216302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c648be4-f9bc-4fda-900f-05ca2b134d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:30.250089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:30.294290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3c648be4-f9bc-4fda-900f-05ca2b134d09 None None] MFA Rules not processed for user `a0d2552bd0e244dea55eb1eecf471fef`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:54:30.295526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3c648be4-f9bc-4fda-900f-05ca2b134d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:30.370925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 757/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Sat Jul 30 04:54:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:54:30.378951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624497f-0002-4b5c-b627-c635034d4629 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:30.380477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c624497f-0002-4b5c-b627-c635034d4629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:30.429749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:30.484588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624497f-0002-4b5c-b627-c635034d4629 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:30.557400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:30.558017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:30.558290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:30.558591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:30.560218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:30.658337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:30.659710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:30.663175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:30.679289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 748/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:54:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:54:30.724527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8eb89-1be3-4443-a001-3b9907f38602 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:30.724820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8eb89-1be3-4443-a001-3b9907f38602 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:30.725092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8eb89-1be3-4443-a001-3b9907f38602 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:30.726387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 758/1506] 10.222.0.20 () {52 vars in 830 bytes} [Sat Jul 30 04:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:54:30.730481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:30.731812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:30.831412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198fa82-02cf-4ea4-9183-07810fda1011 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:30.927271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:30.927920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:30.928085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:30.928358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] PATH_INFO: `/v3/users/c6ac3002dc024090a91c29c571f0ac35` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:30.941064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ac3002dc024090a91c29c571f0ac35)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:30.942095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:54:30.942095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 30 04:54:30.942095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: Jul 30 04:54:30.943329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:30.952881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:31.009462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ac3002dc024090a91c29c571f0ac35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:31.023988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ac3002dc024090a91c29c571f0ac35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:31.025037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ac3002dc024090a91c29c571f0ac35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:31.025569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ac3002dc024090a91c29c571f0ac35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:31.049014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ac3002dc024090a91c29c571f0ac35'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:31.064010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 749/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:54:30 2022] DELETE /identity/v3/users/c6ac3002dc024090a91c29c571f0ac35 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:54:31.068470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6d75c-45f7-4970-ab38-f89fbb6767fb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:31.068951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6d75c-45f7-4970-ab38-f89fbb6767fb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:31.069114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6d75c-45f7-4970-ab38-f89fbb6767fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:31.126413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-14c6d75c-45f7-4970-ab38-f89fbb6767fb None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:54:31.127643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-14c6d75c-45f7-4970-ab38-f89fbb6767fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:31.204063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:31.217007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 759/1508] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:54:31.227273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:31.228544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:31.327749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60e7a3-7839-451d-bc82-1c719dfa96d1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:31.382230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:31.403095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54dd327-607e-4086-b7c9-959cd1ef91ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:31.403712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd327-607e-4086-b7c9-959cd1ef91ab service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:31.403988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd327-607e-4086-b7c9-959cd1ef91ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:31.404259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54dd327-607e-4086-b7c9-959cd1ef91ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:31.405887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b54dd327-607e-4086-b7c9-959cd1ef91ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:31.518142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dd327-607e-4086-b7c9-959cd1ef91ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:31.519839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54dd327-607e-4086-b7c9-959cd1ef91ab service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:31.539152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 750/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:54:31.892225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6d75c-45f7-4970-ab38-f89fbb6767fb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:54:31.893476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-14c6d75c-45f7-4970-ab38-f89fbb6767fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:54:31.905275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6d75c-45f7-4970-ab38-f89fbb6767fb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:54:32.002943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:54:32.003586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:54:32.003865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:54:32.004140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] PATH_INFO: `/v3/projects/d34c8aa4cc754824b9e9f116dbd66819` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:54:32.017196 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34c8aa4cc754824b9e9f116dbd66819)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:54:32.018315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:54:32.018315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 30 04:54:32.018315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: Jul 30 04:54:32.019616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:54:32.078636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34c8aa4cc754824b9e9f116dbd66819', 'request_id': 'req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:32.087549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34c8aa4cc754824b9e9f116dbd66819', 'request_id': 'req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:32.093533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34c8aa4cc754824b9e9f116dbd66819', 'request_id': 'req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:32.123745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] The token cache is being invalidate because project d34c8aa4cc754824b9e9f116dbd66819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:54:32.124034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:32.124550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-557de6f3-b5d5-4f1a-8a52-7f370e603cd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:54:32.125879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 760/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:54:31 2022] DELETE /identity/v3/projects/d34c8aa4cc754824b9e9f116dbd66819 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:54:41.536845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=62 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:54:42.121728 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:56:10.156536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46fca-6070-4d7e-aaaa-044e727183b6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:10.157120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46fca-6070-4d7e-aaaa-044e727183b6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:10.157120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46fca-6070-4d7e-aaaa-044e727183b6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:10.158487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:56:10.163368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47982f9-f0fd-43a7-acfe-7dce285f681a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:10.163797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47982f9-f0fd-43a7-acfe-7dce285f681a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:10.163985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47982f9-f0fd-43a7-acfe-7dce285f681a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:10.246774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e47982f9-f0fd-43a7-acfe-7dce285f681a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:10.247999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e47982f9-f0fd-43a7-acfe-7dce285f681a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:10.353511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:10.510202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46fca-6070-4d7e-aaaa-044e727183b6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:56:10.512347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1ac46fca-6070-4d7e-aaaa-044e727183b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:10.629449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46fca-6070-4d7e-aaaa-044e727183b6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:56:10.634952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41981e-8c1f-4b31-b9cc-551f44758442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:56:10.635563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41981e-8c1f-4b31-b9cc-551f44758442 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:10.635842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41981e-8c1f-4b31-b9cc-551f44758442 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:10.636248 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41981e-8c1f-4b31-b9cc-551f44758442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:10.644556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41981e-8c1f-4b31-b9cc-551f44758442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:56:10.646251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41981e-8c1f-4b31-b9cc-551f44758442 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:56:10.657131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 752/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:56:11.488332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e1f82-14b2-4b52-a3a3-b30a856874e5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:11.488887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e1f82-14b2-4b52-a3a3-b30a856874e5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:11.488887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e1f82-14b2-4b52-a3a3-b30a856874e5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:11.490253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:56:11.495160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2f7dc-2879-4e36-8a43-8a190c6713ec None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:11.495606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2f7dc-2879-4e36-8a43-8a190c6713ec None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:11.495713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2f7dc-2879-4e36-8a43-8a190c6713ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:11.515475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-36e2f7dc-2879-4e36-8a43-8a190c6713ec None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:11.516669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-36e2f7dc-2879-4e36-8a43-8a190c6713ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:11.524620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:11.678834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e1f82-14b2-4b52-a3a3-b30a856874e5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:56:11.693827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e1f82-14b2-4b52-a3a3-b30a856874e5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:56:11.699388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1793c0e-98d0-4223-a21b-e65a6777969b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:56:11.700074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1793c0e-98d0-4223-a21b-e65a6777969b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:11.700354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1793c0e-98d0-4223-a21b-e65a6777969b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:11.700631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1793c0e-98d0-4223-a21b-e65a6777969b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:11.709307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1793c0e-98d0-4223-a21b-e65a6777969b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:56:11.710938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1793c0e-98d0-4223-a21b-e65a6777969b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:56:11.721095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 763/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:56:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:56:12.114250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a8824-d36e-468f-8c8d-220578c83ebe None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:12.114556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a8824-d36e-468f-8c8d-220578c83ebe None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:12.114829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a8824-d36e-468f-8c8d-220578c83ebe None None] PATH_INFO: `/v3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:12.116117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 754/1517] 10.222.0.49 () {54 vars in 925 bytes} [Sat Jul 30 04:56:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:56:12.122319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959599be-1332-4fe1-bd53-5ca7991311fb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:12.122649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959599be-1332-4fe1-bd53-5ca7991311fb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:12.122932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959599be-1332-4fe1-bd53-5ca7991311fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:12.183119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-959599be-1332-4fe1-bd53-5ca7991311fb None None] MFA Rules not processed for user `21b7c64636bd47e6b9a486b6cdaab4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:12.184516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-959599be-1332-4fe1-bd53-5ca7991311fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:12.201973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:56:12.257216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 764/1518] 10.222.0.49 () {58 vars in 1015 bytes} [Sat Jul 30 04:56:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:16.041454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a94e5-a58a-47dc-acb2-865df02de7fd None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:16.041908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a94e5-a58a-47dc-acb2-865df02de7fd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:16.042018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a94e5-a58a-47dc-acb2-865df02de7fd None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:16.043328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 755/1519] 10.222.0.20 () {54 vars in 961 bytes} [Sat Jul 30 04:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:56:16.049368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a70eb-011e-471d-8ba9-be129ee85383 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:16.049777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a70eb-011e-471d-8ba9-be129ee85383 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:16.049929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a70eb-011e-471d-8ba9-be129ee85383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:16.071638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-776a70eb-011e-471d-8ba9-be129ee85383 None None] MFA Rules not processed for user `21b7c64636bd47e6b9a486b6cdaab4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:16.072845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-776a70eb-011e-471d-8ba9-be129ee85383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:16.081831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 765/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Sat Jul 30 04:56:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:16.089097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a94e5-a58a-47dc-acb2-865df02de7fd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:56:16.102395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a94e5-a58a-47dc-acb2-865df02de7fd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:56:16.107852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c4691-838f-4ff0-8f6f-b73ead388db7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:56:16.108443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c4691-838f-4ff0-8f6f-b73ead388db7 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:16.108718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c4691-838f-4ff0-8f6f-b73ead388db7 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:16.108987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c4691-838f-4ff0-8f6f-b73ead388db7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:16.117084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c4691-838f-4ff0-8f6f-b73ead388db7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:56:16.118675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c4691-838f-4ff0-8f6f-b73ead388db7 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:56:16.129321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 756/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:56:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:56:22.944994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ee73c-1715-49a2-aa9d-dc4cebf74025 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:22.945614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ee73c-1715-49a2-aa9d-dc4cebf74025 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:22.945614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ee73c-1715-49a2-aa9d-dc4cebf74025 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:22.946943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:56:22.952688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a473-570f-435d-86f6-90ab29077b2d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:22.953145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a473-570f-435d-86f6-90ab29077b2d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:22.953346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8a473-570f-435d-86f6-90ab29077b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:22.972349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9ce8a473-570f-435d-86f6-90ab29077b2d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:22.973533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9ce8a473-570f-435d-86f6-90ab29077b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:22.981302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:23.134804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ee73c-1715-49a2-aa9d-dc4cebf74025 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:56:23.149526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ee73c-1715-49a2-aa9d-dc4cebf74025 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:56:23.155291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653764e4-e54e-4938-ac7c-b9f8f4ee9a8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:56:23.155899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653764e4-e54e-4938-ac7c-b9f8f4ee9a8a service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:23.156181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653764e4-e54e-4938-ac7c-b9f8f4ee9a8a service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:23.156455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653764e4-e54e-4938-ac7c-b9f8f4ee9a8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:23.164529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653764e4-e54e-4938-ac7c-b9f8f4ee9a8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:56:23.166109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653764e4-e54e-4938-ac7c-b9f8f4ee9a8a service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:56:23.176710 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 767/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:56:24.000355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06deb18b-26c0-401e-ab6b-82e4a7b9a5b7 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:24.000939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06deb18b-26c0-401e-ab6b-82e4a7b9a5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:24.000939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06deb18b-26c0-401e-ab6b-82e4a7b9a5b7 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:24.002273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 30 04:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:56:24.007267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebef4c7-5744-408b-aa6e-a4c2102fbdce None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:24.007702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebef4c7-5744-408b-aa6e-a4c2102fbdce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:24.007805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebef4c7-5744-408b-aa6e-a4c2102fbdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:24.027735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7ebef4c7-5744-408b-aa6e-a4c2102fbdce None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:24.029650 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7ebef4c7-5744-408b-aa6e-a4c2102fbdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:24.037783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 30 04:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:24.189025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06deb18b-26c0-401e-ab6b-82e4a7b9a5b7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:56:24.203912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06deb18b-26c0-401e-ab6b-82e4a7b9a5b7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:56:24.209287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b76471-15d2-4e44-b841-c32d19f6a0df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:56:24.209887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b76471-15d2-4e44-b841-c32d19f6a0df service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:24.210163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b76471-15d2-4e44-b841-c32d19f6a0df service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:24.210451 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b76471-15d2-4e44-b841-c32d19f6a0df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:24.218583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b76471-15d2-4e44-b841-c32d19f6a0df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:56:24.220155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b76471-15d2-4e44-b841-c32d19f6a0df service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:56:24.229659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 759/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:56:28.015539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41bc10-c60d-4015-9e28-a0e820187b29 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:28.016413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41bc10-c60d-4015-9e28-a0e820187b29 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:28.016413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41bc10-c60d-4015-9e28-a0e820187b29 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:28.018924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:56:28.024486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1da825-99c6-4da9-b100-01e0694132df None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:28.025010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1da825-99c6-4da9-b100-01e0694132df None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:28.025160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1da825-99c6-4da9-b100-01e0694132df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:28.047381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2e1da825-99c6-4da9-b100-01e0694132df None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:28.050198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2e1da825-99c6-4da9-b100-01e0694132df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:28.061622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:28.158282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41bc10-c60d-4015-9e28-a0e820187b29 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:56:28.160124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f41bc10-c60d-4015-9e28-a0e820187b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:28.258990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41bc10-c60d-4015-9e28-a0e820187b29 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:56:28.262534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7670ae5-6602-4086-a3ce-c6f20dd8e988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:56:28.263657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7670ae5-6602-4086-a3ce-c6f20dd8e988 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:28.263949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7670ae5-6602-4086-a3ce-c6f20dd8e988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:28.264199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7670ae5-6602-4086-a3ce-c6f20dd8e988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:28.273094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7670ae5-6602-4086-a3ce-c6f20dd8e988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:56:28.274484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7670ae5-6602-4086-a3ce-c6f20dd8e988 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:56:28.284348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 770/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:56:30.278041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ae45-12fa-4ad5-a893-9293e58b7298 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:30.279145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ae45-12fa-4ad5-a893-9293e58b7298 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:30.279145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ae45-12fa-4ad5-a893-9293e58b7298 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:30.280423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:56:30.285664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8101268-32b1-46ea-8309-cd0358829f42 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:30.286126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8101268-32b1-46ea-8309-cd0358829f42 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:30.286242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8101268-32b1-46ea-8309-cd0358829f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:30.307092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c8101268-32b1-46ea-8309-cd0358829f42 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:30.308302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c8101268-32b1-46ea-8309-cd0358829f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:30.316643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:30.352290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ae45-12fa-4ad5-a893-9293e58b7298 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:56:30.363785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ae45-12fa-4ad5-a893-9293e58b7298 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:56:30.367324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956bc2d-d9b2-4755-8809-f0210446d389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:56:30.367924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956bc2d-d9b2-4755-8809-f0210446d389 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:30.368207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956bc2d-d9b2-4755-8809-f0210446d389 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:30.368479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956bc2d-d9b2-4755-8809-f0210446d389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:30.377927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956bc2d-d9b2-4755-8809-f0210446d389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:56:30.379450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956bc2d-d9b2-4755-8809-f0210446d389 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:56:30.389207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 762/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:56:33.754955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adaf90-ad63-45a2-a825-e59b4845b15e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:33.755537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adaf90-ad63-45a2-a825-e59b4845b15e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:33.755537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adaf90-ad63-45a2-a825-e59b4845b15e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:33.756864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:56:33.761896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67150ca-45dd-4648-a6d4-c1a68770b253 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:33.762314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67150ca-45dd-4648-a6d4-c1a68770b253 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:33.762448 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67150ca-45dd-4648-a6d4-c1a68770b253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:33.770940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=81 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:56:33.784291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c67150ca-45dd-4648-a6d4-c1a68770b253 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:33.785491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c67150ca-45dd-4648-a6d4-c1a68770b253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:33.793356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:34.097651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adaf90-ad63-45a2-a825-e59b4845b15e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:56:34.099344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2adaf90-ad63-45a2-a825-e59b4845b15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:34.164370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:56:34.179162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adaf90-ad63-45a2-a825-e59b4845b15e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:56:34.182759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9c657-bf48-4d96-b36a-86a940911203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:56:34.183370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9c657-bf48-4d96-b36a-86a940911203 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:34.183647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9c657-bf48-4d96-b36a-86a940911203 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:34.183919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9c657-bf48-4d96-b36a-86a940911203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:34.192925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9c657-bf48-4d96-b36a-86a940911203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:56:34.194340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9c657-bf48-4d96-b36a-86a940911203 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:56:34.203712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 773/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:56:36.225135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13ec43-506f-4852-b4b8-13d3c61623ab None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:36.225660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13ec43-506f-4852-b4b8-13d3c61623ab None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:36.225660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13ec43-506f-4852-b4b8-13d3c61623ab None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:36.227180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 30 04:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:56:36.232998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46280f-132f-42b4-ad5d-4c43921af4d8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:36.233425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46280f-132f-42b4-ad5d-4c43921af4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:36.233536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46280f-132f-42b4-ad5d-4c43921af4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:36.253445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ae46280f-132f-42b4-ad5d-4c43921af4d8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:56:36.254673 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae46280f-132f-42b4-ad5d-4c43921af4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:36.262584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 30 04:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:56:36.563003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13ec43-506f-4852-b4b8-13d3c61623ab None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:56:36.564689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0c13ec43-506f-4852-b4b8-13d3c61623ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:56:36.578271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13ec43-506f-4852-b4b8-13d3c61623ab None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:56:36.581789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21f162-d0d7-4927-8998-5ab26b155d44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:56:36.582390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21f162-d0d7-4927-8998-5ab26b155d44 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:56:36.582701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21f162-d0d7-4927-8998-5ab26b155d44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:56:36.582972 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21f162-d0d7-4927-8998-5ab26b155d44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:56:36.592446 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21f162-d0d7-4927-8998-5ab26b155d44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:56:36.593859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21f162-d0d7-4927-8998-5ab26b155d44 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:56:36.603206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 765/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:56:46.248440 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=11 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:56:46.599105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=8 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:03.833376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d24837-6a65-4fd6-bf83-e7bd1a960db0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:03.833986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d24837-6a65-4fd6-bf83-e7bd1a960db0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:03.833986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d24837-6a65-4fd6-bf83-e7bd1a960db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:03.867564 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-16d24837-6a65-4fd6-bf83-e7bd1a960db0 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:03.868794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16d24837-6a65-4fd6-bf83-e7bd1a960db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:03.879378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 775/1540] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:03.888866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125647b-510a-4fe9-a7cb-1a49c667c2e5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:03.889940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125647b-510a-4fe9-a7cb-1a49c667c2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:03.890481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125647b-510a-4fe9-a7cb-1a49c667c2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:03.918468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9125647b-510a-4fe9-a7cb-1a49c667c2e5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:03.920476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9125647b-510a-4fe9-a7cb-1a49c667c2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:03.933223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 766/1541] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:03.938179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d24837-6a65-4fd6-bf83-e7bd1a960db0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:03.952532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b2f5c-7ac2-428d-a00f-452cca302b19 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:03.953170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b2f5c-7ac2-428d-a00f-452cca302b19 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:03.953570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d24837-6a65-4fd6-bf83-e7bd1a960db0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:03.953927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b2f5c-7ac2-428d-a00f-452cca302b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:03.961766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d16eaa-73d6-4e03-ac3c-4300ce1f854b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:03.962558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d16eaa-73d6-4e03-ac3c-4300ce1f854b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:03.963566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d16eaa-73d6-4e03-ac3c-4300ce1f854b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:03.963841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d16eaa-73d6-4e03-ac3c-4300ce1f854b admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:03.964542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d16eaa-73d6-4e03-ac3c-4300ce1f854b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:03.964843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d16eaa-73d6-4e03-ac3c-4300ce1f854b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:03.967404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d16eaa-73d6-4e03-ac3c-4300ce1f854b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:03.984955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5e6b2f5c-7ac2-428d-a00f-452cca302b19 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:03.986595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5e6b2f5c-7ac2-428d-a00f-452cca302b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:03.993057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 776/1542] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:03.994501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74beb2-b495-47b5-9976-aa7a7028852f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:03.994954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74beb2-b495-47b5-9976-aa7a7028852f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:03.995238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74beb2-b495-47b5-9976-aa7a7028852f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:03.999237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 767/1543] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.025362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93a8f-f08d-4ec2-856f-51657d177f47 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.026411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93a8f-f08d-4ec2-856f-51657d177f47 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.026956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93a8f-f08d-4ec2-856f-51657d177f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.030058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2c74beb2-b495-47b5-9976-aa7a7028852f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.031325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2c74beb2-b495-47b5-9976-aa7a7028852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.051985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 777/1544] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.052912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74beb2-b495-47b5-9976-aa7a7028852f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.068549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74beb2-b495-47b5-9976-aa7a7028852f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.070034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e5b93a8f-f08d-4ec2-856f-51657d177f47 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.071611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e5b93a8f-f08d-4ec2-856f-51657d177f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.077097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d2b5a-4b79-4175-8e2e-76d1ab332280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.078304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d2b5a-4b79-4175-8e2e-76d1ab332280 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.078612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d2b5a-4b79-4175-8e2e-76d1ab332280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.078887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d2b5a-4b79-4175-8e2e-76d1ab332280 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.079662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d2b5a-4b79-4175-8e2e-76d1ab332280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.080719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-f15d2b5a-4b79-4175-8e2e-76d1ab332280 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:58:04.080719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 30 04:58:04.080719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: Jul 30 04:58:04.081911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d2b5a-4b79-4175-8e2e-76d1ab332280 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.089108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 768/1545] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.090742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876622ee-237c-47e0-a884-457811363b98 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.091261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876622ee-237c-47e0-a884-457811363b98 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.091759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876622ee-237c-47e0-a884-457811363b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.131540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-876622ee-237c-47e0-a884-457811363b98 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.132959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-876622ee-237c-47e0-a884-457811363b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.141896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 769/1546] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.143404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876622ee-237c-47e0-a884-457811363b98 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.143541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 778/1547] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/projects => generated 389 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.144809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf238aa-e061-47f6-b565-0f216b53c541 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.145104 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf238aa-e061-47f6-b565-0f216b53c541 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.145400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf238aa-e061-47f6-b565-0f216b53c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.156824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876622ee-237c-47e0-a884-457811363b98 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.172996 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7cf238aa-e061-47f6-b565-0f216b53c541 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.174285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7cf238aa-e061-47f6-b565-0f216b53c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.219729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9bb8d-5a26-4740-bcfc-b55446b80f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.220350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9bb8d-5a26-4740-bcfc-b55446b80f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.220623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9bb8d-5a26-4740-bcfc-b55446b80f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.220897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9bb8d-5a26-4740-bcfc-b55446b80f91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.221583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9bb8d-5a26-4740-bcfc-b55446b80f91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:04.221884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9bb8d-5a26-4740-bcfc-b55446b80f91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.222932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-f9c9bb8d-5a26-4740-bcfc-b55446b80f91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:58:04.222932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 30 04:58:04.222932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: Jul 30 04:58:04.223807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9bb8d-5a26-4740-bcfc-b55446b80f91 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.234814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 779/1548] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.242213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 770/1549] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:04.242410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff99776-691d-4e79-9bde-aca4aafec21a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.242410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff99776-691d-4e79-9bde-aca4aafec21a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.242410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff99776-691d-4e79-9bde-aca4aafec21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.243247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e1360-03f7-4dd6-85c6-c6f8c6d888e0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.243247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e1360-03f7-4dd6-85c6-c6f8c6d888e0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.243247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e1360-03f7-4dd6-85c6-c6f8c6d888e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.259042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d45e1360-03f7-4dd6-85c6-c6f8c6d888e0 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.260649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bff99776-691d-4e79-9bde-aca4aafec21a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.260899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d45e1360-03f7-4dd6-85c6-c6f8c6d888e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.261913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bff99776-691d-4e79-9bde-aca4aafec21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.268942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 771/1550] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.269888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 780/1551] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.270125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8d41f-b7c5-4fbe-ad08-fa99eed53f7f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.270410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8d41f-b7c5-4fbe-ad08-fa99eed53f7f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.270714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8d41f-b7c5-4fbe-ad08-fa99eed53f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.275976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff99776-691d-4e79-9bde-aca4aafec21a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.286308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff99776-691d-4e79-9bde-aca4aafec21a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.290544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-eca8d41f-b7c5-4fbe-ad08-fa99eed53f7f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.291045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c323c4-1fa9-45a4-9d6c-dcd978caf720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.291859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c323c4-1fa9-45a4-9d6c-dcd978caf720 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.292006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eca8d41f-b7c5-4fbe-ad08-fa99eed53f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.292140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c323c4-1fa9-45a4-9d6c-dcd978caf720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.292418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c323c4-1fa9-45a4-9d6c-dcd978caf720 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.293132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c323c4-1fa9-45a4-9d6c-dcd978caf720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.295144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c323c4-1fa9-45a4-9d6c-dcd978caf720 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.300216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 772/1552] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.300984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8d41f-b7c5-4fbe-ad08-fa99eed53f7f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.313173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8d41f-b7c5-4fbe-ad08-fa99eed53f7f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.321093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 781/1553] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/users => generated 295 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.321251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9fde9-235c-45d0-8a5e-46874da04475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.321251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9fde9-235c-45d0-8a5e-46874da04475 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.321251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9fde9-235c-45d0-8a5e-46874da04475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.321561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4665ab-9404-454a-be6e-8a36e1004df3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.321692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9fde9-235c-45d0-8a5e-46874da04475 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.321786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4665ab-9404-454a-be6e-8a36e1004df3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.321786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4665ab-9404-454a-be6e-8a36e1004df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.321948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9fde9-235c-45d0-8a5e-46874da04475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:04.321948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9fde9-235c-45d0-8a5e-46874da04475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.323314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9fde9-235c-45d0-8a5e-46874da04475 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.336475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 773/1554] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:04.338237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7673614-0c9d-4f2f-b9c9-83739ab77b5b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.338675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7673614-0c9d-4f2f-b9c9-83739ab77b5b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.338991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7673614-0c9d-4f2f-b9c9-83739ab77b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.342019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7a4665ab-9404-454a-be6e-8a36e1004df3 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.343269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7a4665ab-9404-454a-be6e-8a36e1004df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.351523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 782/1555] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.352442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4665ab-9404-454a-be6e-8a36e1004df3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.362431 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4665ab-9404-454a-be6e-8a36e1004df3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.364982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c7673614-0c9d-4f2f-b9c9-83739ab77b5b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.366226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c7673614-0c9d-4f2f-b9c9-83739ab77b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.367112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f482b-e272-4c02-9955-faaed782e702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.367710 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f482b-e272-4c02-9955-faaed782e702 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.367991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f482b-e272-4c02-9955-faaed782e702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.368326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f482b-e272-4c02-9955-faaed782e702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.368940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f482b-e272-4c02-9955-faaed782e702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.370978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f482b-e272-4c02-9955-faaed782e702 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.374551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 774/1556] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.375675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982294a-52bd-451b-9378-6411f768cf90 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.375954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982294a-52bd-451b-9378-6411f768cf90 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.376228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982294a-52bd-451b-9378-6411f768cf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.395710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d982294a-52bd-451b-9378-6411f768cf90 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.396915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d982294a-52bd-451b-9378-6411f768cf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.402004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 783/1557] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.403302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60649183-1462-49b0-a3b0-a57376718e83 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.403593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60649183-1462-49b0-a3b0-a57376718e83 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.404008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60649183-1462-49b0-a3b0-a57376718e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.409504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 775/1558] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.409504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c500f8-eb9b-4765-b045-bf9ae163f514 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.409504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c500f8-eb9b-4765-b045-bf9ae163f514 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.409504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c500f8-eb9b-4765-b045-bf9ae163f514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.424491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-60649183-1462-49b0-a3b0-a57376718e83 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.425713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60649183-1462-49b0-a3b0-a57376718e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.427889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f0c500f8-eb9b-4765-b045-bf9ae163f514 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.429037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f0c500f8-eb9b-4765-b045-bf9ae163f514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.470683 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:04.486948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 784/1559] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.487760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60649183-1462-49b0-a3b0-a57376718e83 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.493200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 776/1560] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.494214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c500f8-eb9b-4765-b045-bf9ae163f514 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.497089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60649183-1462-49b0-a3b0-a57376718e83 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.501661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccb6ba-f177-4ef9-816c-fe4e0426cb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.502268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb6ba-f177-4ef9-816c-fe4e0426cb3c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.502577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb6ba-f177-4ef9-816c-fe4e0426cb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.502848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccb6ba-f177-4ef9-816c-fe4e0426cb3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.503531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c500f8-eb9b-4765-b045-bf9ae163f514 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.503649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccb6ba-f177-4ef9-816c-fe4e0426cb3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.505129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ccb6ba-f177-4ef9-816c-fe4e0426cb3c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.508364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b491d-7b36-4782-9d51-b800d58b8134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.508958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b491d-7b36-4782-9d51-b800d58b8134 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.509234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b491d-7b36-4782-9d51-b800d58b8134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.509550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b491d-7b36-4782-9d51-b800d58b8134 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.510224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b491d-7b36-4782-9d51-b800d58b8134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.512141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b491d-7b36-4782-9d51-b800d58b8134 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.515445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 785/1561] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:04.516627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5586e-7234-4c91-a5a7-007c98b88b98 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.516910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5586e-7234-4c91-a5a7-007c98b88b98 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.517178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5586e-7234-4c91-a5a7-007c98b88b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.537695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-23e5586e-7234-4c91-a5a7-007c98b88b98 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:04.538928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23e5586e-7234-4c91-a5a7-007c98b88b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:04.542140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 777/1562] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.546347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b491d-7b36-4782-9d51-b800d58b8134 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.554870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:04.605023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 786/1563] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:04.606256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5586e-7234-4c91-a5a7-007c98b88b98 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.614220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b491d-7b36-4782-9d51-b800d58b8134 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.618048 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5586e-7234-4c91-a5a7-007c98b88b98 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.619098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777b541-3457-40b8-bdc3-7d353462836c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.619712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777b541-3457-40b8-bdc3-7d353462836c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.619988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777b541-3457-40b8-bdc3-7d353462836c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.620259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777b541-3457-40b8-bdc3-7d353462836c admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.620960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777b541-3457-40b8-bdc3-7d353462836c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:04.621262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777b541-3457-40b8-bdc3-7d353462836c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.622917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777b541-3457-40b8-bdc3-7d353462836c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.623069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f411228-9fde-4401-9aea-05dac2545bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.623682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f411228-9fde-4401-9aea-05dac2545bdb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.623965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f411228-9fde-4401-9aea-05dac2545bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.624247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f411228-9fde-4401-9aea-05dac2545bdb admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.624911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f411228-9fde-4401-9aea-05dac2545bdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.626866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f411228-9fde-4401-9aea-05dac2545bdb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.640353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 778/1564] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:04.640353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777b541-3457-40b8-bdc3-7d353462836c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.647593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777b541-3457-40b8-bdc3-7d353462836c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.653882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ace5fe-fe70-4fbe-b022-b9ae11f8aa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.654518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ace5fe-fe70-4fbe-b022-b9ae11f8aa50 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.654796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ace5fe-fe70-4fbe-b022-b9ae11f8aa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.655066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ace5fe-fe70-4fbe-b022-b9ae11f8aa50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.655745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ace5fe-fe70-4fbe-b022-b9ae11f8aa50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:04.656039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ace5fe-fe70-4fbe-b022-b9ae11f8aa50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.657365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 787/1565] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/users => generated 287 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.657620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ace5fe-fe70-4fbe-b022-b9ae11f8aa50 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.658290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f411228-9fde-4401-9aea-05dac2545bdb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.669293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f411228-9fde-4401-9aea-05dac2545bdb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.671086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 779/1566] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:04.674356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ca1fb-38da-43d1-b7ae-9785c683cbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.674988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ca1fb-38da-43d1-b7ae-9785c683cbef admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.675271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ca1fb-38da-43d1-b7ae-9785c683cbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.675570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ca1fb-38da-43d1-b7ae-9785c683cbef admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.676268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ca1fb-38da-43d1-b7ae-9785c683cbef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:04.676570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ca1fb-38da-43d1-b7ae-9785c683cbef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.678266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ca1fb-38da-43d1-b7ae-9785c683cbef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.678509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ace5fe-fe70-4fbe-b022-b9ae11f8aa50 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.688173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ace5fe-fe70-4fbe-b022-b9ae11f8aa50 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.696704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 788/1567] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:04.696704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ca1fb-38da-43d1-b7ae-9785c683cbef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.699096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90b886-5435-4f88-99dc-b2b784a64fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.699716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90b886-5435-4f88-99dc-b2b784a64fca admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.699997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90b886-5435-4f88-99dc-b2b784a64fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.700274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90b886-5435-4f88-99dc-b2b784a64fca admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.700964 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90b886-5435-4f88-99dc-b2b784a64fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:04.701265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90b886-5435-4f88-99dc-b2b784a64fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.702927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90b886-5435-4f88-99dc-b2b784a64fca admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.703908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ca1fb-38da-43d1-b7ae-9785c683cbef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.708612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8c620-004c-4f75-ade4-c4e2bed6bd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.709205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8c620-004c-4f75-ade4-c4e2bed6bd47 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.709481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8c620-004c-4f75-ade4-c4e2bed6bd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.709754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8c620-004c-4f75-ade4-c4e2bed6bd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.710451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8c620-004c-4f75-ade4-c4e2bed6bd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.712016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8c620-004c-4f75-ade4-c4e2bed6bd47 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.715820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 780/1568] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:04.716613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90b886-5435-4f88-99dc-b2b784a64fca admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.726300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90b886-5435-4f88-99dc-b2b784a64fca admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.729450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 789/1569] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:04.730343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8c620-004c-4f75-ade4-c4e2bed6bd47 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.734375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3809aee-d1b6-41f4-8cea-119efdfa1101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.735105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3809aee-d1b6-41f4-8cea-119efdfa1101 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.735395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3809aee-d1b6-41f4-8cea-119efdfa1101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.735662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3809aee-d1b6-41f4-8cea-119efdfa1101 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.736358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3809aee-d1b6-41f4-8cea-119efdfa1101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.737390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-f3809aee-d1b6-41f4-8cea-119efdfa1101 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:58:04.737390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 30 04:58:04.737390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: Jul 30 04:58:04.738620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3809aee-d1b6-41f4-8cea-119efdfa1101 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.742015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8c620-004c-4f75-ade4-c4e2bed6bd47 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.747541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2fff9-5dfb-4166-89d9-fcfe8093d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.748506 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2fff9-5dfb-4166-89d9-fcfe8093d155 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.748506 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2fff9-5dfb-4166-89d9-fcfe8093d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.748734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2fff9-5dfb-4166-89d9-fcfe8093d155 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.749370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2fff9-5dfb-4166-89d9-fcfe8093d155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:04.749678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2fff9-5dfb-4166-89d9-fcfe8093d155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.751328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2fff9-5dfb-4166-89d9-fcfe8093d155 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.762013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 781/1570] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.763398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3809aee-d1b6-41f4-8cea-119efdfa1101 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.768016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 790/1571] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:04.769060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2fff9-5dfb-4166-89d9-fcfe8093d155 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.780341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2fff9-5dfb-4166-89d9-fcfe8093d155 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.782096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3809aee-d1b6-41f4-8cea-119efdfa1101 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.785807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf20ff7-2292-40e6-9e1a-eb7f76caa486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.786588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf20ff7-2292-40e6-9e1a-eb7f76caa486 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.786875 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf20ff7-2292-40e6-9e1a-eb7f76caa486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.787154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf20ff7-2292-40e6-9e1a-eb7f76caa486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.787578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8e500-7e0f-4adb-98c9-beab67e3a470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.787849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf20ff7-2292-40e6-9e1a-eb7f76caa486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.788193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8e500-7e0f-4adb-98c9-beab67e3a470 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.788470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8e500-7e0f-4adb-98c9-beab67e3a470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.788742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8e500-7e0f-4adb-98c9-beab67e3a470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.789402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8e500-7e0f-4adb-98c9-beab67e3a470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.789507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf20ff7-2292-40e6-9e1a-eb7f76caa486 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.791907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8e500-7e0f-4adb-98c9-beab67e3a470 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.800694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 791/1572] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:04.801558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf20ff7-2292-40e6-9e1a-eb7f76caa486 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.812810 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf20ff7-2292-40e6-9e1a-eb7f76caa486 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.818009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c3e4c-c62c-45c8-8888-206accb801eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.818668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c3e4c-c62c-45c8-8888-206accb801eb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.818957 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c3e4c-c62c-45c8-8888-206accb801eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.819230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c3e4c-c62c-45c8-8888-206accb801eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.819892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c3e4c-c62c-45c8-8888-206accb801eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.821806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c3e4c-c62c-45c8-8888-206accb801eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.822660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 782/1573] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.823558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8e500-7e0f-4adb-98c9-beab67e3a470 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.853904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 792/1574] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.855230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c3e4c-c62c-45c8-8888-206accb801eb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.891974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8e500-7e0f-4adb-98c9-beab67e3a470 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.906040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:04.918228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c3e4c-c62c-45c8-8888-206accb801eb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:04.923642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bd4cd-ad84-40e4-9896-dec09e16c218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.924257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bd4cd-ad84-40e4-9896-dec09e16c218 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.924537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bd4cd-ad84-40e4-9896-dec09e16c218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.924809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bd4cd-ad84-40e4-9896-dec09e16c218 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.925462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bd4cd-ad84-40e4-9896-dec09e16c218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.927461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bd4cd-ad84-40e4-9896-dec09e16c218 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.936547 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:04.954857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3d7e1-2a56-415d-a5d2-e951fcac4733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:04.955120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 793/1575] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.955479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3d7e1-2a56-415d-a5d2-e951fcac4733 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:04.955755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3d7e1-2a56-415d-a5d2-e951fcac4733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:04.956024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3d7e1-2a56-415d-a5d2-e951fcac4733 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:04.956685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3d7e1-2a56-415d-a5d2-e951fcac4733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:04.957089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bd4cd-ad84-40e4-9896-dec09e16c218 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:04.958614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3d7e1-2a56-415d-a5d2-e951fcac4733 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:04.985849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 783/1576] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:04 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:04.986704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3d7e1-2a56-415d-a5d2-e951fcac4733 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.034540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bd4cd-ad84-40e4-9896-dec09e16c218 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.048769 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3d7e1-2a56-415d-a5d2-e951fcac4733 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.053482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd82d42-9527-4f6f-85c9-bf48f714ac9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.054096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd82d42-9527-4f6f-85c9-bf48f714ac9e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.054402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd82d42-9527-4f6f-85c9-bf48f714ac9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.054705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd82d42-9527-4f6f-85c9-bf48f714ac9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.055405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd82d42-9527-4f6f-85c9-bf48f714ac9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.057191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd82d42-9527-4f6f-85c9-bf48f714ac9e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.067132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 784/1577] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.067906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd82d42-9527-4f6f-85c9-bf48f714ac9e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.077390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd82d42-9527-4f6f-85c9-bf48f714ac9e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.082009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18dc97-3ba4-4514-9220-374ba9c448ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.082619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18dc97-3ba4-4514-9220-374ba9c448ec admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.082896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18dc97-3ba4-4514-9220-374ba9c448ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.083168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18dc97-3ba4-4514-9220-374ba9c448ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.083818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18dc97-3ba4-4514-9220-374ba9c448ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.085672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18dc97-3ba4-4514-9220-374ba9c448ec admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.095873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a66821-a653-4fc2-9d7e-1c6fc1b3428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.096553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a66821-a653-4fc2-9d7e-1c6fc1b3428a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.096835 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a66821-a653-4fc2-9d7e-1c6fc1b3428a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.097114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a66821-a653-4fc2-9d7e-1c6fc1b3428a admin admin] PATH_INFO: `/v3/projects/bc6f8afc1abf427f92c44fdc55994b18/users/d3a56c6d44b74429a6616cdfc2869b71/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.113257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a66821-a653-4fc2-9d7e-1c6fc1b3428a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6f8afc1abf427f92c44fdc55994b18, user_id=d3a56c6d44b74429a6616cdfc2869b71, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.113716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 785/1578] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:05 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:05.114275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-50a66821-a653-4fc2-9d7e-1c6fc1b3428a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:58:05.114275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 30 04:58:05.114275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: Jul 30 04:58:05.114758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18dc97-3ba4-4514-9220-374ba9c448ec admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.116322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a66821-a653-4fc2-9d7e-1c6fc1b3428a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.129852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 794/1579] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:04 2022] PUT /identity/v3/projects/bc6f8afc1abf427f92c44fdc55994b18/users/d3a56c6d44b74429a6616cdfc2869b71/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:05.130680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a66821-a653-4fc2-9d7e-1c6fc1b3428a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.185736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18dc97-3ba4-4514-9220-374ba9c448ec admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.190275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3c5f6-9c5d-4ace-b0d5-6ecf57f927e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.190918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3c5f6-9c5d-4ace-b0d5-6ecf57f927e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.191197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3c5f6-9c5d-4ace-b0d5-6ecf57f927e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.191496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3c5f6-9c5d-4ace-b0d5-6ecf57f927e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.192183 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3c5f6-9c5d-4ace-b0d5-6ecf57f927e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.193793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3c5f6-9c5d-4ace-b0d5-6ecf57f927e6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.193914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a66821-a653-4fc2-9d7e-1c6fc1b3428a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.198794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e119225-f301-4e87-a6a8-ca580115c12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.199417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e119225-f301-4e87-a6a8-ca580115c12e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.199717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e119225-f301-4e87-a6a8-ca580115c12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.199982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e119225-f301-4e87-a6a8-ca580115c12e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.200645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e119225-f301-4e87-a6a8-ca580115c12e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.202582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e119225-f301-4e87-a6a8-ca580115c12e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.203857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 786/1580] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.204628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3c5f6-9c5d-4ace-b0d5-6ecf57f927e6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.214592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3c5f6-9c5d-4ace-b0d5-6ecf57f927e6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.219585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:05.220184 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd93a6d-2591-4cab-92f5-ece93ca85815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.220633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd93a6d-2591-4cab-92f5-ece93ca85815 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.221056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd93a6d-2591-4cab-92f5-ece93ca85815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.221301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd93a6d-2591-4cab-92f5-ece93ca85815 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.221974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd93a6d-2591-4cab-92f5-ece93ca85815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.223952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd93a6d-2591-4cab-92f5-ece93ca85815 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.227185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 795/1581] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:05 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:05.228030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e119225-f301-4e87-a6a8-ca580115c12e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.240550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e119225-f301-4e87-a6a8-ca580115c12e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.245857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b29025-e96b-47ee-9876-c8b7d3f1c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.246476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b29025-e96b-47ee-9876-c8b7d3f1c30e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.246850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b29025-e96b-47ee-9876-c8b7d3f1c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.247090 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b29025-e96b-47ee-9876-c8b7d3f1c30e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.247670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b29025-e96b-47ee-9876-c8b7d3f1c30e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.249556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b29025-e96b-47ee-9876-c8b7d3f1c30e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.249678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 787/1582] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:05 2022] POST /identity/v3/users => generated 302 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:05.250427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd93a6d-2591-4cab-92f5-ece93ca85815 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.260267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd93a6d-2591-4cab-92f5-ece93ca85815 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.266787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8eb5c-47ca-45c3-b7e8-12f092f0669a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.267414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8eb5c-47ca-45c3-b7e8-12f092f0669a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.267809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8eb5c-47ca-45c3-b7e8-12f092f0669a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.268106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8eb5c-47ca-45c3-b7e8-12f092f0669a admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.268761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8eb5c-47ca-45c3-b7e8-12f092f0669a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.270716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8eb5c-47ca-45c3-b7e8-12f092f0669a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.273389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 796/1583] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:05 2022] POST /identity/v3/users => generated 283 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:05.274317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b29025-e96b-47ee-9876-c8b7d3f1c30e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.285487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b29025-e96b-47ee-9876-c8b7d3f1c30e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.290629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9e0ec-b080-4493-ac9f-f67693a3a3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.292026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e0ec-b080-4493-ac9f-f67693a3a3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.292354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e0ec-b080-4493-ac9f-f67693a3a3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.292632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e0ec-b080-4493-ac9f-f67693a3a3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.293349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9e0ec-b080-4493-ac9f-f67693a3a3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.293842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 788/1584] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:05 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:05.294636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8eb5c-47ca-45c3-b7e8-12f092f0669a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.294971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9e0ec-b080-4493-ac9f-f67693a3a3f7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.299942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:05.306104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8eb5c-47ca-45c3-b7e8-12f092f0669a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.307077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 797/1585] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.308631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22785823-a680-453a-b6b3-c29b29413e30 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.308925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22785823-a680-453a-b6b3-c29b29413e30 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.309205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22785823-a680-453a-b6b3-c29b29413e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.311925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181fb6a-4780-4196-a816-0c40f95429cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.312553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181fb6a-4780-4196-a816-0c40f95429cb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.312803 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181fb6a-4780-4196-a816-0c40f95429cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.313073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181fb6a-4780-4196-a816-0c40f95429cb admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.313775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181fb6a-4780-4196-a816-0c40f95429cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.315827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181fb6a-4780-4196-a816-0c40f95429cb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.329269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-22785823-a680-453a-b6b3-c29b29413e30 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:05.330751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-22785823-a680-453a-b6b3-c29b29413e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:05.339997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 789/1586] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:05 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:05.343885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181fb6a-4780-4196-a816-0c40f95429cb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.344063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 798/1587] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:05.344063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22785823-a680-453a-b6b3-c29b29413e30 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.354823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22785823-a680-453a-b6b3-c29b29413e30 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.359430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af208e-56b7-4ee4-b24c-3d1d93208006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.359600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181fb6a-4780-4196-a816-0c40f95429cb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.360067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af208e-56b7-4ee4-b24c-3d1d93208006 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.360343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af208e-56b7-4ee4-b24c-3d1d93208006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.360662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af208e-56b7-4ee4-b24c-3d1d93208006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.361287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af208e-56b7-4ee4-b24c-3d1d93208006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.362910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af208e-56b7-4ee4-b24c-3d1d93208006 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.364313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48df8d2-073b-4489-ab5e-3b2b207bd869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.364917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48df8d2-073b-4489-ab5e-3b2b207bd869 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.365227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48df8d2-073b-4489-ab5e-3b2b207bd869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.365506 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48df8d2-073b-4489-ab5e-3b2b207bd869 admin admin] PATH_INFO: `/v3/projects/0a7dc50349e84acba8f9476de06cd75d/users/5b3d5b5b3c1743a6ac5af0174a04db5e/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.373624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 799/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.375167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af208e-56b7-4ee4-b24c-3d1d93208006 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.381001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48df8d2-073b-4489-ab5e-3b2b207bd869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7dc50349e84acba8f9476de06cd75d, user_id=5b3d5b5b3c1743a6ac5af0174a04db5e, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.383865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48df8d2-073b-4489-ab5e-3b2b207bd869 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.385124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af208e-56b7-4ee4-b24c-3d1d93208006 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.390342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1d639-bb94-49aa-8c88-6624aea252d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.391128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1d639-bb94-49aa-8c88-6624aea252d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.391499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1d639-bb94-49aa-8c88-6624aea252d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.391781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1d639-bb94-49aa-8c88-6624aea252d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.392473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1d639-bb94-49aa-8c88-6624aea252d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.394193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1d639-bb94-49aa-8c88-6624aea252d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.398671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 790/1589] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:05 2022] PUT /identity/v3/projects/0a7dc50349e84acba8f9476de06cd75d/users/5b3d5b5b3c1743a6ac5af0174a04db5e/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:05.399404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48df8d2-073b-4489-ab5e-3b2b207bd869 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.406470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 800/1590] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.406637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1d639-bb94-49aa-8c88-6624aea252d2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.417652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1d639-bb94-49aa-8c88-6624aea252d2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.469224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48df8d2-073b-4489-ab5e-3b2b207bd869 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.474024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370dad8-9ca8-49a4-a597-67743ae8620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.474666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370dad8-9ca8-49a4-a597-67743ae8620a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.474941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370dad8-9ca8-49a4-a597-67743ae8620a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.475207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370dad8-9ca8-49a4-a597-67743ae8620a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.475324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c47b6c-69d0-4ad1-85b5-48d635012627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.475846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c47b6c-69d0-4ad1-85b5-48d635012627 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.476042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370dad8-9ca8-49a4-a597-67743ae8620a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.476159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c47b6c-69d0-4ad1-85b5-48d635012627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.476377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c47b6c-69d0-4ad1-85b5-48d635012627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.477060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c47b6c-69d0-4ad1-85b5-48d635012627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.477796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370dad8-9ca8-49a4-a597-67743ae8620a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.478707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c47b6c-69d0-4ad1-85b5-48d635012627 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.487842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 791/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.491771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370dad8-9ca8-49a4-a597-67743ae8620a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.491908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 801/1592] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.491908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c47b6c-69d0-4ad1-85b5-48d635012627 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.500962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370dad8-9ca8-49a4-a597-67743ae8620a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.501223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c47b6c-69d0-4ad1-85b5-48d635012627 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.505427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b90814-a03e-4d12-b137-70dfc7f923ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.505669 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c01b6b-20ed-446e-9f61-658c49f2becd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.506068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b90814-a03e-4d12-b137-70dfc7f923ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.506249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c01b6b-20ed-446e-9f61-658c49f2becd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.506385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b90814-a03e-4d12-b137-70dfc7f923ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.506557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c01b6b-20ed-446e-9f61-658c49f2becd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.506709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b90814-a03e-4d12-b137-70dfc7f923ba admin admin] PATH_INFO: `/v3/projects/ef5fe5ac675540c0930bd7bab0d8b27f/users/691a9c5e4d534dcc9c26f8cf88cd7fb5/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.506820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c01b6b-20ed-446e-9f61-658c49f2becd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.507490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c01b6b-20ed-446e-9f61-658c49f2becd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.509054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c01b6b-20ed-446e-9f61-658c49f2becd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.520083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 802/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.520880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c01b6b-20ed-446e-9f61-658c49f2becd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.525937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b90814-a03e-4d12-b137-70dfc7f923ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef5fe5ac675540c0930bd7bab0d8b27f, user_id=691a9c5e4d534dcc9c26f8cf88cd7fb5, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.528862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b90814-a03e-4d12-b137-70dfc7f923ba admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.543209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 792/1594] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:05 2022] PUT /identity/v3/projects/ef5fe5ac675540c0930bd7bab0d8b27f/users/691a9c5e4d534dcc9c26f8cf88cd7fb5/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:05.543767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b90814-a03e-4d12-b137-70dfc7f923ba admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.576419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c01b6b-20ed-446e-9f61-658c49f2becd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.610666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b90814-a03e-4d12-b137-70dfc7f923ba admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.616153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:05.617259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07caca-bdd5-4fbd-a4ed-909498b63900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.618069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07caca-bdd5-4fbd-a4ed-909498b63900 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.618387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07caca-bdd5-4fbd-a4ed-909498b63900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.618751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07caca-bdd5-4fbd-a4ed-909498b63900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.619821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07caca-bdd5-4fbd-a4ed-909498b63900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.621507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07caca-bdd5-4fbd-a4ed-909498b63900 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.626640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf1acc-1cd1-4cda-b6e7-f3f5a8f251e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.627265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf1acc-1cd1-4cda-b6e7-f3f5a8f251e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.627546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf1acc-1cd1-4cda-b6e7-f3f5a8f251e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.627938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf1acc-1cd1-4cda-b6e7-f3f5a8f251e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.631737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 793/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.632534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07caca-bdd5-4fbd-a4ed-909498b63900 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.638273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf1acc-1cd1-4cda-b6e7-f3f5a8f251e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.639751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf1acc-1cd1-4cda-b6e7-f3f5a8f251e1 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.642407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07caca-bdd5-4fbd-a4ed-909498b63900 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.647379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10620d13-4215-475d-83c9-e80ab186e340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.648223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10620d13-4215-475d-83c9-e80ab186e340 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.648609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10620d13-4215-475d-83c9-e80ab186e340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.648990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10620d13-4215-475d-83c9-e80ab186e340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.649460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 803/1596] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:05.651404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee266a-cafb-4fc5-8ab5-e8ed558dcb5b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.651691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee266a-cafb-4fc5-8ab5-e8ed558dcb5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.651995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee266a-cafb-4fc5-8ab5-e8ed558dcb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.655216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10620d13-4215-475d-83c9-e80ab186e340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.657379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10620d13-4215-475d-83c9-e80ab186e340 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.669462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 794/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.670537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10620d13-4215-475d-83c9-e80ab186e340 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.680899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10620d13-4215-475d-83c9-e80ab186e340 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.684591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-45ee266a-cafb-4fc5-8ab5-e8ed558dcb5b None None] MFA Rules not processed for user `5b3d5b5b3c1743a6ac5af0174a04db5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:05.685949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-45ee266a-cafb-4fc5-8ab5-e8ed558dcb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:05.687581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc4752-f9f5-4f2e-b2ae-f818ab24d9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.688629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc4752-f9f5-4f2e-b2ae-f818ab24d9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.689000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc4752-f9f5-4f2e-b2ae-f818ab24d9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.689429 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc4752-f9f5-4f2e-b2ae-f818ab24d9e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.690269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc4752-f9f5-4f2e-b2ae-f818ab24d9e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.692108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc4752-f9f5-4f2e-b2ae-f818ab24d9e8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.703789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 795/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.707624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc4752-f9f5-4f2e-b2ae-f818ab24d9e8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.718857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc4752-f9f5-4f2e-b2ae-f818ab24d9e8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.724355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e60c0-bfd2-44bb-9bff-2a00d59d9525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.725166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e60c0-bfd2-44bb-9bff-2a00d59d9525 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.725553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e60c0-bfd2-44bb-9bff-2a00d59d9525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.725936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e60c0-bfd2-44bb-9bff-2a00d59d9525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.726782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e60c0-bfd2-44bb-9bff-2a00d59d9525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.729055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e60c0-bfd2-44bb-9bff-2a00d59d9525 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.730832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:05.746765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 796/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.748322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e60c0-bfd2-44bb-9bff-2a00d59d9525 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.762809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e60c0-bfd2-44bb-9bff-2a00d59d9525 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.768896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb994c-68cd-40dd-a0cf-76612a62a702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.769639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb994c-68cd-40dd-a0cf-76612a62a702 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.770049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb994c-68cd-40dd-a0cf-76612a62a702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.773622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 804/1600] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:05.773781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb994c-68cd-40dd-a0cf-76612a62a702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.773781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb994c-68cd-40dd-a0cf-76612a62a702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.773968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a229b6e-2d6a-4bdb-924b-1277cab96f23 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.773968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a229b6e-2d6a-4bdb-924b-1277cab96f23 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.773968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a229b6e-2d6a-4bdb-924b-1277cab96f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.774235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb994c-68cd-40dd-a0cf-76612a62a702 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.788853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 797/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:05.789907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb994c-68cd-40dd-a0cf-76612a62a702 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.794777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4a229b6e-2d6a-4bdb-924b-1277cab96f23 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:05.796766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a229b6e-2d6a-4bdb-924b-1277cab96f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:05.801759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb994c-68cd-40dd-a0cf-76612a62a702 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.809321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 805/1602] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:05.809486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d4c99-5d12-4832-abba-55fa423251ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.809588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a229b6e-2d6a-4bdb-924b-1277cab96f23 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.809674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d4c99-5d12-4832-abba-55fa423251ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.809674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d4c99-5d12-4832-abba-55fa423251ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.809674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d4c99-5d12-4832-abba-55fa423251ca admin admin] PATH_INFO: `/v3/projects/922209a39e114d2dbd25c771d269631e/users/ad49f82a91914d978f3cc15e1b7e0c29/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.825212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a229b6e-2d6a-4bdb-924b-1277cab96f23 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.827193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d4c99-5d12-4832-abba-55fa423251ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=922209a39e114d2dbd25c771d269631e, user_id=ad49f82a91914d978f3cc15e1b7e0c29, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.830025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d4c99-5d12-4832-abba-55fa423251ca admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.832068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5a296-b396-4834-b7bf-e455561498dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.833011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5a296-b396-4834-b7bf-e455561498dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.833401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5a296-b396-4834-b7bf-e455561498dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.833761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5a296-b396-4834-b7bf-e455561498dc admin admin] PATH_INFO: `/v3/projects/e479b62b596746669c1a6d8470c4faed/users/fae22984127147bd9b31c3853960bc6f/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:05.857280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5a296-b396-4834-b7bf-e455561498dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e479b62b596746669c1a6d8470c4faed, user_id=fae22984127147bd9b31c3853960bc6f, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:05.860026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5a296-b396-4834-b7bf-e455561498dc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:05.863889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 798/1603] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:05 2022] PUT /identity/v3/projects/922209a39e114d2dbd25c771d269631e/users/ad49f82a91914d978f3cc15e1b7e0c29/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:05.864560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d4c99-5d12-4832-abba-55fa423251ca admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.887476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 806/1604] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:05 2022] PUT /identity/v3/projects/e479b62b596746669c1a6d8470c4faed/users/fae22984127147bd9b31c3853960bc6f/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:05.888350 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5a296-b396-4834-b7bf-e455561498dc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:05.950945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d4c99-5d12-4832-abba-55fa423251ca admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.975635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5a296-b396-4834-b7bf-e455561498dc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:05.981490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef40d59-e81c-4205-82ba-9a195db64c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:05.982363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef40d59-e81c-4205-82ba-9a195db64c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:05.982842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef40d59-e81c-4205-82ba-9a195db64c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:05.983277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef40d59-e81c-4205-82ba-9a195db64c10 admin admin] PATH_INFO: `/v3/projects/e01500cb03c74d1082090cd1c1836169/users/a8abb58f9c274e568d2d0fe3d4df11de/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.004279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef40d59-e81c-4205-82ba-9a195db64c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01500cb03c74d1082090cd1c1836169, user_id=a8abb58f9c274e568d2d0fe3d4df11de, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:06.016473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef40d59-e81c-4205-82ba-9a195db64c10 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:06.020576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb364-b38e-4417-a5f8-1af723b30da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:06.021413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb364-b38e-4417-a5f8-1af723b30da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.021696 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb364-b38e-4417-a5f8-1af723b30da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.021971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb364-b38e-4417-a5f8-1af723b30da2 admin admin] PATH_INFO: `/v3/projects/c9e7f1507317483cb11d635432823ce6/users/96e88c9f3e374d7d89f4d2a0a1efc3a5/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.039289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 807/1605] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:05 2022] PUT /identity/v3/projects/e01500cb03c74d1082090cd1c1836169/users/a8abb58f9c274e568d2d0fe3d4df11de/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:06.043414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef40d59-e81c-4205-82ba-9a195db64c10 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:06.046630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bb364-b38e-4417-a5f8-1af723b30da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e7f1507317483cb11d635432823ce6, user_id=96e88c9f3e374d7d89f4d2a0a1efc3a5, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:06.049957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bb364-b38e-4417-a5f8-1af723b30da2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:06.081865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 799/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:05 2022] PUT /identity/v3/projects/c9e7f1507317483cb11d635432823ce6/users/96e88c9f3e374d7d89f4d2a0a1efc3a5/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:06.083018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb364-b38e-4417-a5f8-1af723b30da2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:06.092580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:06.135748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef40d59-e81c-4205-82ba-9a195db64c10 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:06.143547 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:06.160681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb364-b38e-4417-a5f8-1af723b30da2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:06.167741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe5079-9aa5-4b8b-b491-7fc084d041f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:06.168998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe5079-9aa5-4b8b-b491-7fc084d041f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.169287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe5079-9aa5-4b8b-b491-7fc084d041f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.169751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe5079-9aa5-4b8b-b491-7fc084d041f9 admin admin] PATH_INFO: `/v3/projects/decb0bd64582440590890900e594b79e/users/25db6f7402ae4a14a04fab0a9915ab77/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.191778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe5079-9aa5-4b8b-b491-7fc084d041f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=decb0bd64582440590890900e594b79e, user_id=25db6f7402ae4a14a04fab0a9915ab77, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:06.194963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe5079-9aa5-4b8b-b491-7fc084d041f9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:06.196839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0d361-5744-4970-bd54-ad787f2d57e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:06.198038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0d361-5744-4970-bd54-ad787f2d57e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.198317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0d361-5744-4970-bd54-ad787f2d57e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.198616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0d361-5744-4970-bd54-ad787f2d57e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.199279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0d361-5744-4970-bd54-ad787f2d57e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:06.201395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0d361-5744-4970-bd54-ad787f2d57e6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:06.211120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 800/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:06 2022] PUT /identity/v3/projects/decb0bd64582440590890900e594b79e/users/25db6f7402ae4a14a04fab0a9915ab77/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:06.212320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe5079-9aa5-4b8b-b491-7fc084d041f9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:06.216414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-12fe5079-9aa5-4b8b-b491-7fc084d041f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:06.235776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 808/1608] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/projects => generated 371 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:06.240872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e434c5-8a96-47da-89af-07d844d9bbc3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.241172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e434c5-8a96-47da-89af-07d844d9bbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.241448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e434c5-8a96-47da-89af-07d844d9bbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.273651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d9e434c5-8a96-47da-89af-07d844d9bbc3 None None] MFA Rules not processed for user `ad49f82a91914d978f3cc15e1b7e0c29`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:06.274615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9e434c5-8a96-47da-89af-07d844d9bbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:06.282209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe5079-9aa5-4b8b-b491-7fc084d041f9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:06.346908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcef3e-6ac5-4ad9-a0ba-c24f7253410f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:06.347367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcef3e-6ac5-4ad9-a0ba-c24f7253410f service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.347781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcef3e-6ac5-4ad9-a0ba-c24f7253410f service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.348062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcef3e-6ac5-4ad9-a0ba-c24f7253410f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.371391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 809/1609] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:06.372823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fc508-57c4-41bc-82ab-bd0798cd0ec0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.373188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fc508-57c4-41bc-82ab-bd0798cd0ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.373476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fc508-57c4-41bc-82ab-bd0798cd0ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.431652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f15fc508-57c4-41bc-82ab-bd0798cd0ec0 None None] MFA Rules not processed for user `fae22984127147bd9b31c3853960bc6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:06.434808 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f15fc508-57c4-41bc-82ab-bd0798cd0ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:06.446038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcef3e-6ac5-4ad9-a0ba-c24f7253410f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:06.448301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcef3e-6ac5-4ad9-a0ba-c24f7253410f service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:06.469522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 801/1610] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:06.471899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc643b3-f55c-4054-b7a0-c93c3c711e5e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.472194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc643b3-f55c-4054-b7a0-c93c3c711e5e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.472479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc643b3-f55c-4054-b7a0-c93c3c711e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.488716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:06.506670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cdc643b3-f55c-4054-b7a0-c93c3c711e5e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:06.515535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cdc643b3-f55c-4054-b7a0-c93c3c711e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:06.528834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 810/1611] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:06.530348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e7c71-e252-4a01-821b-d875a437fcc3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.530808 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e7c71-e252-4a01-821b-d875a437fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.531193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e7c71-e252-4a01-821b-d875a437fcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.537077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 802/1612] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:06.538520 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281835-cf71-4452-8886-e447e2c01e92 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.538811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281835-cf71-4452-8886-e447e2c01e92 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.539094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281835-cf71-4452-8886-e447e2c01e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.570616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7d7e7c71-e252-4a01-821b-d875a437fcc3 None None] MFA Rules not processed for user `96e88c9f3e374d7d89f4d2a0a1efc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:06.572669 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d7e7c71-e252-4a01-821b-d875a437fcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:06.578802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ba281835-cf71-4452-8886-e447e2c01e92 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:06.580069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ba281835-cf71-4452-8886-e447e2c01e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:06.601599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 803/1613] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:06.605703 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba281835-cf71-4452-8886-e447e2c01e92 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:06.628530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba281835-cf71-4452-8886-e447e2c01e92 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:06.646802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bba7d5-8507-4490-87f6-5bdc3072ea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:06.647449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bba7d5-8507-4490-87f6-5bdc3072ea86 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.647729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bba7d5-8507-4490-87f6-5bdc3072ea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.648080 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bba7d5-8507-4490-87f6-5bdc3072ea86 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.648694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bba7d5-8507-4490-87f6-5bdc3072ea86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:06.650882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bba7d5-8507-4490-87f6-5bdc3072ea86 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:06.660444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 811/1614] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:06.662131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ab961-d201-4e6e-9dec-0386cf0ba5fd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.662417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ab961-d201-4e6e-9dec-0386cf0ba5fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.662816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ab961-d201-4e6e-9dec-0386cf0ba5fd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.665543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 812/1615] 10.222.0.20 () {54 vars in 962 bytes} [Sat Jul 30 04:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:06.665543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ab961-d201-4e6e-9dec-0386cf0ba5fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:06.689159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ab961-d201-4e6e-9dec-0386cf0ba5fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:06.696005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae41715-b4d0-4ccc-b3e6-630440fe0e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:06.696669 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae41715-b4d0-4ccc-b3e6-630440fe0e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.698220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae41715-b4d0-4ccc-b3e6-630440fe0e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.698418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae41715-b4d0-4ccc-b3e6-630440fe0e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.699109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae41715-b4d0-4ccc-b3e6-630440fe0e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:06.701285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae41715-b4d0-4ccc-b3e6-630440fe0e37 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:06.701414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 804/1616] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:06.702154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bba7d5-8507-4490-87f6-5bdc3072ea86 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:06.721084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bba7d5-8507-4490-87f6-5bdc3072ea86 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:06.726052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae175b-a60e-45c0-bf1d-06a0dee6ad9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:06.727250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae175b-a60e-45c0-bf1d-06a0dee6ad9c service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.727613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae175b-a60e-45c0-bf1d-06a0dee6ad9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.728030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae175b-a60e-45c0-bf1d-06a0dee6ad9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.738380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:06.742429 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae175b-a60e-45c0-bf1d-06a0dee6ad9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:06.746508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae175b-a60e-45c0-bf1d-06a0dee6ad9c service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:06.750820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 813/1617] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/projects => generated 403 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:06.756144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae41715-b4d0-4ccc-b3e6-630440fe0e37 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:06.774879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 805/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:06.779498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0360b4-56c7-4c63-b543-44174caa465c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.780718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0360b4-56c7-4c63-b543-44174caa465c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.781079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0360b4-56c7-4c63-b543-44174caa465c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.849041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae41715-b4d0-4ccc-b3e6-630440fe0e37 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:06.856197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da657a-f95d-43ea-83fe-f1b24cd039e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:06.859530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da657a-f95d-43ea-83fe-f1b24cd039e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:06.860635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da657a-f95d-43ea-83fe-f1b24cd039e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:06.861216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da657a-f95d-43ea-83fe-f1b24cd039e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:06.862174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da657a-f95d-43ea-83fe-f1b24cd039e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:06.864609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da657a-f95d-43ea-83fe-f1b24cd039e9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:06.884251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cc0360b4-56c7-4c63-b543-44174caa465c None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:06.886542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cc0360b4-56c7-4c63-b543-44174caa465c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:06.906306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 814/1619] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/projects => generated 363 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:06.907645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da657a-f95d-43ea-83fe-f1b24cd039e9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:06.961765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:07.008807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da657a-f95d-43ea-83fe-f1b24cd039e9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.023288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12cef9-2ee7-4e83-9dca-3ad04c01b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.024369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12cef9-2ee7-4e83-9dca-3ad04c01b1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.024469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 806/1620] 10.222.0.20 () {58 vars in 1059 bytes} [Sat Jul 30 04:58:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:07.024844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12cef9-2ee7-4e83-9dca-3ad04c01b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.024844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12cef9-2ee7-4e83-9dca-3ad04c01b1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.026350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0360b4-56c7-4c63-b543-44174caa465c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.027814 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12cef9-2ee7-4e83-9dca-3ad04c01b1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.033914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12cef9-2ee7-4e83-9dca-3ad04c01b1d8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.065117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 815/1621] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:07.065117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12cef9-2ee7-4e83-9dca-3ad04c01b1d8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.084890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12cef9-2ee7-4e83-9dca-3ad04c01b1d8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.094799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b99ce-3703-4f07-9b04-02d2db6ab032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.096444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b99ce-3703-4f07-9b04-02d2db6ab032 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.096444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b99ce-3703-4f07-9b04-02d2db6ab032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.096444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b99ce-3703-4f07-9b04-02d2db6ab032 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.096958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b99ce-3703-4f07-9b04-02d2db6ab032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.098835 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b99ce-3703-4f07-9b04-02d2db6ab032 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.135073 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 816/1622] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:07 2022] POST /identity/v3/users => generated 308 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:07.136141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b99ce-3703-4f07-9b04-02d2db6ab032 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.150051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b99ce-3703-4f07-9b04-02d2db6ab032 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.158739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecc1b9-7237-416a-bb63-05dc885548a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.159253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecc1b9-7237-416a-bb63-05dc885548a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.159635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecc1b9-7237-416a-bb63-05dc885548a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.159904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecc1b9-7237-416a-bb63-05dc885548a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.160020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0360b4-56c7-4c63-b543-44174caa465c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.161322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecc1b9-7237-416a-bb63-05dc885548a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.163922 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecc1b9-7237-416a-bb63-05dc885548a5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.201258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 817/1623] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:07 2022] POST /identity/v3/users => generated 288 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:07.206802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecc1b9-7237-416a-bb63-05dc885548a5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.302292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3ba91-4f97-4eeb-b55b-4d549cbd1584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.302976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3ba91-4f97-4eeb-b55b-4d549cbd1584 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.303263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3ba91-4f97-4eeb-b55b-4d549cbd1584 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.303537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3ba91-4f97-4eeb-b55b-4d549cbd1584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.319697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3ba91-4f97-4eeb-b55b-4d549cbd1584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.321308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3ba91-4f97-4eeb-b55b-4d549cbd1584 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.324226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecc1b9-7237-416a-bb63-05dc885548a5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.346960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19146ad-41e5-4db0-a3c7-3ddca938e366 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.347842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19146ad-41e5-4db0-a3c7-3ddca938e366 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.348213 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19146ad-41e5-4db0-a3c7-3ddca938e366 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.348566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19146ad-41e5-4db0-a3c7-3ddca938e366 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.349835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 807/1624] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:07.350836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3ba91-4f97-4eeb-b55b-4d549cbd1584 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.363508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3ba91-4f97-4eeb-b55b-4d549cbd1584 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.369727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce02ce-5dae-4bfa-943c-47b4fbe58fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.370577 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce02ce-5dae-4bfa-943c-47b4fbe58fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.371028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce02ce-5dae-4bfa-943c-47b4fbe58fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.371377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce02ce-5dae-4bfa-943c-47b4fbe58fa7 admin admin] PATH_INFO: `/v3/projects/5719fd43ba514a39b397de32494363f8/users/b9392b17abed4984ad8d064507e243e8/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.395649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce02ce-5dae-4bfa-943c-47b4fbe58fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5719fd43ba514a39b397de32494363f8, user_id=b9392b17abed4984ad8d064507e243e8, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.398813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce02ce-5dae-4bfa-943c-47b4fbe58fa7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.421671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 808/1625] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:07 2022] PUT /identity/v3/projects/5719fd43ba514a39b397de32494363f8/users/b9392b17abed4984ad8d064507e243e8/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:07.425127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce02ce-5dae-4bfa-943c-47b4fbe58fa7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.440515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:07.531374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce02ce-5dae-4bfa-943c-47b4fbe58fa7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.532667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19146ad-41e5-4db0-a3c7-3ddca938e366 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.535000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19146ad-41e5-4db0-a3c7-3ddca938e366 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.542335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdde74-5a6a-4b2f-924a-8d1d20d39dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.543705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdde74-5a6a-4b2f-924a-8d1d20d39dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.544097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdde74-5a6a-4b2f-924a-8d1d20d39dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.544478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccdde74-5a6a-4b2f-924a-8d1d20d39dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.548057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdde74-5a6a-4b2f-924a-8d1d20d39dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.550351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccdde74-5a6a-4b2f-924a-8d1d20d39dcd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.565353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 818/1626] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:07.569190 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19146ad-41e5-4db0-a3c7-3ddca938e366 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.569521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 809/1627] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:07.569521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdde74-5a6a-4b2f-924a-8d1d20d39dcd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.581738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccdde74-5a6a-4b2f-924a-8d1d20d39dcd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.587349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfdd18-dd0e-4fcf-9787-f602742308c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.587349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfdd18-dd0e-4fcf-9787-f602742308c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.587912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfdd18-dd0e-4fcf-9787-f602742308c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.588468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfdd18-dd0e-4fcf-9787-f602742308c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.589470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfdd18-dd0e-4fcf-9787-f602742308c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.591462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfdd18-dd0e-4fcf-9787-f602742308c4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.598260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19146ad-41e5-4db0-a3c7-3ddca938e366 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.605006 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77110b9-f816-45bd-9fb5-5f21485c6f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.605599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77110b9-f816-45bd-9fb5-5f21485c6f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.605927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77110b9-f816-45bd-9fb5-5f21485c6f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.606200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77110b9-f816-45bd-9fb5-5f21485c6f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.606994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77110b9-f816-45bd-9fb5-5f21485c6f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.608423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 810/1628] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:07.608647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77110b9-f816-45bd-9fb5-5f21485c6f11 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.609521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfdd18-dd0e-4fcf-9787-f602742308c4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.623578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfdd18-dd0e-4fcf-9787-f602742308c4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.630739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 819/1629] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:07.632526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77110b9-f816-45bd-9fb5-5f21485c6f11 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.633344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545987de-13ed-4e14-b982-cf6a5ff2732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.635487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545987de-13ed-4e14-b982-cf6a5ff2732a admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.635944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545987de-13ed-4e14-b982-cf6a5ff2732a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.636763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545987de-13ed-4e14-b982-cf6a5ff2732a admin admin] PATH_INFO: `/v3/projects/b5bc85452c4a4e37ad3b9c0800f05e46/users/20afe06e885e451f8e7c79b1ceb6eee9/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.657217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77110b9-f816-45bd-9fb5-5f21485c6f11 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.668898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcce76-7ded-436a-878e-e652a859c0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.670033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcce76-7ded-436a-878e-e652a859c0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.671202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcce76-7ded-436a-878e-e652a859c0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.672884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545987de-13ed-4e14-b982-cf6a5ff2732a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bc85452c4a4e37ad3b9c0800f05e46, user_id=20afe06e885e451f8e7c79b1ceb6eee9, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.673484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcce76-7ded-436a-878e-e652a859c0e9 admin admin] PATH_INFO: `/v3/projects/5719fd43ba514a39b397de32494363f8/users/b9392b17abed4984ad8d064507e243e8/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.677458 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545987de-13ed-4e14-b982-cf6a5ff2732a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.680302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcce76-7ded-436a-878e-e652a859c0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5719fd43ba514a39b397de32494363f8, user_id=b9392b17abed4984ad8d064507e243e8, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.686352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcce76-7ded-436a-878e-e652a859c0e9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.711916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 811/1630] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:07 2022] PUT /identity/v3/projects/b5bc85452c4a4e37ad3b9c0800f05e46/users/20afe06e885e451f8e7c79b1ceb6eee9/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:07.713695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545987de-13ed-4e14-b982-cf6a5ff2732a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.727762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 820/1631] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:07 2022] PUT /identity/v3/projects/5719fd43ba514a39b397de32494363f8/users/b9392b17abed4984ad8d064507e243e8/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:07.727762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcce76-7ded-436a-878e-e652a859c0e9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:07.750098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:07.866896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545987de-13ed-4e14-b982-cf6a5ff2732a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.866896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a24d62-bb5e-4179-8691-db0da246fbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:07.869476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a24d62-bb5e-4179-8691-db0da246fbd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.869476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a24d62-bb5e-4179-8691-db0da246fbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.869476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a24d62-bb5e-4179-8691-db0da246fbd8 admin admin] PATH_INFO: `/v3/projects/54d694b49b424dcbaa8349d6605f5e34/users/97f5de3b9a7641c09774c445344c2a1f/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:07.913679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a24d62-bb5e-4179-8691-db0da246fbd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54d694b49b424dcbaa8349d6605f5e34, user_id=97f5de3b9a7641c09774c445344c2a1f, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:07.918534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a24d62-bb5e-4179-8691-db0da246fbd8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:07.951660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 812/1632] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:07 2022] PUT /identity/v3/projects/54d694b49b424dcbaa8349d6605f5e34/users/97f5de3b9a7641c09774c445344c2a1f/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:07.951660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198374a-ff3f-455c-9823-5e90514ac4b5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:07.952161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcce76-7ded-436a-878e-e652a859c0e9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:07.952646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198374a-ff3f-455c-9823-5e90514ac4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:07.953100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198374a-ff3f-455c-9823-5e90514ac4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.060755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4198374a-ff3f-455c-9823-5e90514ac4b5 None None] MFA Rules not processed for user `b9392b17abed4984ad8d064507e243e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:08.064946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4198374a-ff3f-455c-9823-5e90514ac4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:08.071093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfaa4e8-14be-4828-8bed-facadca38587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:08.071752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfaa4e8-14be-4828-8bed-facadca38587 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.075227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfaa4e8-14be-4828-8bed-facadca38587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.075530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfaa4e8-14be-4828-8bed-facadca38587 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.076271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfaa4e8-14be-4828-8bed-facadca38587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:08.078258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfaa4e8-14be-4828-8bed-facadca38587 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:08.095086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 821/1633] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:08.096659 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfaa4e8-14be-4828-8bed-facadca38587 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:08.111194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfaa4e8-14be-4828-8bed-facadca38587 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:08.119737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324280bc-e5b8-4d4c-ad9c-09b74a39cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:08.120473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324280bc-e5b8-4d4c-ad9c-09b74a39cbfb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.121402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324280bc-e5b8-4d4c-ad9c-09b74a39cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.121693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324280bc-e5b8-4d4c-ad9c-09b74a39cbfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.122421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324280bc-e5b8-4d4c-ad9c-09b74a39cbfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:08.125258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324280bc-e5b8-4d4c-ad9c-09b74a39cbfb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:08.146053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:08.147008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 822/1634] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:08.149578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068b208-5b3a-4973-aef2-9907438509fd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.149889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068b208-5b3a-4973-aef2-9907438509fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.151424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068b208-5b3a-4973-aef2-9907438509fd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.154472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 823/1635] 10.222.0.20 () {54 vars in 912 bytes} [Sat Jul 30 04:58:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:08.155539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068b208-5b3a-4973-aef2-9907438509fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:08.176777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068b208-5b3a-4973-aef2-9907438509fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:08.183387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cb1e2-6eb1-4ecf-abb0-ca5a114e6d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:08.184136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb1e2-6eb1-4ecf-abb0-ca5a114e6d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.184425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb1e2-6eb1-4ecf-abb0-ca5a114e6d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.184705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb1e2-6eb1-4ecf-abb0-ca5a114e6d82 admin admin] PATH_INFO: `/v3/projects/b5bc85452c4a4e37ad3b9c0800f05e46/users/20afe06e885e451f8e7c79b1ceb6eee9/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.187485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cb1e2-6eb1-4ecf-abb0-ca5a114e6d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bc85452c4a4e37ad3b9c0800f05e46, user_id=20afe06e885e451f8e7c79b1ceb6eee9, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:08.190167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 813/1636] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:08.191173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cb1e2-6eb1-4ecf-abb0-ca5a114e6d82 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:08.193378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198374a-ff3f-455c-9823-5e90514ac4b5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:08.215990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198374a-ff3f-455c-9823-5e90514ac4b5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:08.229359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 824/1637] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:08 2022] PUT /identity/v3/projects/b5bc85452c4a4e37ad3b9c0800f05e46/users/20afe06e885e451f8e7c79b1ceb6eee9/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:08.229359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c28d9-40db-40d0-94f5-9cfc92142c73 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.229359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c28d9-40db-40d0-94f5-9cfc92142c73 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.229359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c28d9-40db-40d0-94f5-9cfc92142c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.264007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b06c28d9-40db-40d0-94f5-9cfc92142c73 None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:08.265285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b06c28d9-40db-40d0-94f5-9cfc92142c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:08.331680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040c8b9-3be1-4433-ac99-f90389c03477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:08.332865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040c8b9-3be1-4433-ac99-f90389c03477 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.333570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040c8b9-3be1-4433-ac99-f90389c03477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.333910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040c8b9-3be1-4433-ac99-f90389c03477 admin admin] PATH_INFO: `/v3/projects/54d694b49b424dcbaa8349d6605f5e34/users/97f5de3b9a7641c09774c445344c2a1f/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.337051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040c8b9-3be1-4433-ac99-f90389c03477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54d694b49b424dcbaa8349d6605f5e34, user_id=97f5de3b9a7641c09774c445344c2a1f, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:08.340659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040c8b9-3be1-4433-ac99-f90389c03477 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:08.358615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 814/1638] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:08 2022] PUT /identity/v3/projects/54d694b49b424dcbaa8349d6605f5e34/users/97f5de3b9a7641c09774c445344c2a1f/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:08.360010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040c8b9-3be1-4433-ac99-f90389c03477 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:08.401736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 825/1639] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 04:58:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:08.403253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579532c-2ffc-4d47-9c02-4458d598b456 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.403608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579532c-2ffc-4d47-9c02-4458d598b456 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.404221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579532c-2ffc-4d47-9c02-4458d598b456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.458699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7579532c-2ffc-4d47-9c02-4458d598b456 None None] MFA Rules not processed for user `20afe06e885e451f8e7c79b1ceb6eee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:08.458699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7579532c-2ffc-4d47-9c02-4458d598b456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:08.472259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:08.485463 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040c8b9-3be1-4433-ac99-f90389c03477 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:08.490417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935be8ef-8655-4cca-814f-ef7e7df9fcd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:08.495170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935be8ef-8655-4cca-814f-ef7e7df9fcd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.495579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935be8ef-8655-4cca-814f-ef7e7df9fcd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.496013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935be8ef-8655-4cca-814f-ef7e7df9fcd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.607621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 826/1640] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:08.607621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c3431-2633-4f0f-985f-36ffc1ae5688 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.607621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c3431-2633-4f0f-985f-36ffc1ae5688 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.607621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c3431-2633-4f0f-985f-36ffc1ae5688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.614908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935be8ef-8655-4cca-814f-ef7e7df9fcd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:08.617028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935be8ef-8655-4cca-814f-ef7e7df9fcd6 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:08.651363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 815/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:08.653235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935be8ef-8655-4cca-814f-ef7e7df9fcd6 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:08.667768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935be8ef-8655-4cca-814f-ef7e7df9fcd6 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:08.676549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d07b7-b8b2-4099-bf13-c697563726be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:08.677293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d07b7-b8b2-4099-bf13-c697563726be service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.677658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d07b7-b8b2-4099-bf13-c697563726be service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.678099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d07b7-b8b2-4099-bf13-c697563726be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.697970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-222c3431-2633-4f0f-985f-36ffc1ae5688 None None] MFA Rules not processed for user `97f5de3b9a7641c09774c445344c2a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:08.699944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-222c3431-2633-4f0f-985f-36ffc1ae5688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:08.738901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:08.834573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 827/1642] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:08 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:08.836018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c3431-2633-4f0f-985f-36ffc1ae5688 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:08.848497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c3431-2633-4f0f-985f-36ffc1ae5688 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:08.854171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd5735-c5be-4e9f-9db6-13759c76cf40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:08.855388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd5735-c5be-4e9f-9db6-13759c76cf40 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:08.855841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd5735-c5be-4e9f-9db6-13759c76cf40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:08.856265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd5735-c5be-4e9f-9db6-13759c76cf40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:08.867807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d07b7-b8b2-4099-bf13-c697563726be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:08.876650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d07b7-b8b2-4099-bf13-c697563726be service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:08.879779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd5735-c5be-4e9f-9db6-13759c76cf40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:08.883573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd5735-c5be-4e9f-9db6-13759c76cf40 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:08.898117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 816/1643] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:08.903232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 828/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:08 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:09.107030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d07b7-b8b2-4099-bf13-c697563726be service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:09.119522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d07b7-b8b2-4099-bf13-c697563726be service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:09.130363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f6b71-f8d4-410f-991c-8d7f4b88f512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:09.131073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6b71-f8d4-410f-991c-8d7f4b88f512 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:09.131369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6b71-f8d4-410f-991c-8d7f4b88f512 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:09.131621 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6b71-f8d4-410f-991c-8d7f4b88f512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:09.163597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f6b71-f8d4-410f-991c-8d7f4b88f512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:09.165197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f6b71-f8d4-410f-991c-8d7f4b88f512 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:09.189801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 817/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:09.721620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd5735-c5be-4e9f-9db6-13759c76cf40 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:09.902971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd5735-c5be-4e9f-9db6-13759c76cf40 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:09.921346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:09.922519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:09.923988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:09.924636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] PATH_INFO: `/v3/users/fae22984127147bd9b31c3853960bc6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:09.928725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae22984127147bd9b31c3853960bc6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:09.930886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:58:09.930886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 30 04:58:09.930886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: Jul 30 04:58:09.933709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:09.983438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:10.101438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae22984127147bd9b31c3853960bc6f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.132524 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae22984127147bd9b31c3853960bc6f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.133770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae22984127147bd9b31c3853960bc6f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.135336 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae22984127147bd9b31c3853960bc6f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.188153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae22984127147bd9b31c3853960bc6f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.211108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 829/1646] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:09 2022] DELETE /identity/v3/users/fae22984127147bd9b31c3853960bc6f => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:10.216670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0ba07-fe7e-4347-9f18-6da521d90cd0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:10.217174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0ba07-fe7e-4347-9f18-6da521d90cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:10.217575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0ba07-fe7e-4347-9f18-6da521d90cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:10.249303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c5f0ba07-fe7e-4347-9f18-6da521d90cd0 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:10.251224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c5f0ba07-fe7e-4347-9f18-6da521d90cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:10.391483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 818/1647] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:10.406693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:10.408967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:10.480100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0ba07-fe7e-4347-9f18-6da521d90cd0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:10.481665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c5f0ba07-fe7e-4347-9f18-6da521d90cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:10.507617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0ba07-fe7e-4347-9f18-6da521d90cd0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:10.519747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:10.520450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:10.520732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:10.521008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] PATH_INFO: `/v3/users/5b3d5b5b3c1743a6ac5af0174a04db5e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:10.524241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b3d5b5b3c1743a6ac5af0174a04db5e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:10.529501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:10.542550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6236a-73b9-444e-b696-25fba3b428d9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:10.548208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eca87a-0a56-48d0-8c63-fa1fe0cdbf3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:10.548898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eca87a-0a56-48d0-8c63-fa1fe0cdbf3c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:10.549189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eca87a-0a56-48d0-8c63-fa1fe0cdbf3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:10.549475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eca87a-0a56-48d0-8c63-fa1fe0cdbf3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:10.560515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eca87a-0a56-48d0-8c63-fa1fe0cdbf3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:10.562044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eca87a-0a56-48d0-8c63-fa1fe0cdbf3c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:10.575375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 830/1648] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:10.576201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eca87a-0a56-48d0-8c63-fa1fe0cdbf3c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:10.577725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f3eca87a-0a56-48d0-8c63-fa1fe0cdbf3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:10.593084 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eca87a-0a56-48d0-8c63-fa1fe0cdbf3c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:10.598644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:10.599351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:10.601012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:10.601012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] PATH_INFO: `/v3/users/96e88c9f3e374d7d89f4d2a0a1efc3a5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:10.601224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96e88c9f3e374d7d89f4d2a0a1efc3a5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:10.605127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:10.638691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b3d5b5b3c1743a6ac5af0174a04db5e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.656425 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b3d5b5b3c1743a6ac5af0174a04db5e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.657535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b3d5b5b3c1743a6ac5af0174a04db5e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.658627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b3d5b5b3c1743a6ac5af0174a04db5e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.679663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f2cdff92-8abb-46dc-880d-25363e4c22f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b3d5b5b3c1743a6ac5af0174a04db5e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.699417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96e88c9f3e374d7d89f4d2a0a1efc3a5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.715961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 819/1649] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:10 2022] DELETE /identity/v3/users/5b3d5b5b3c1743a6ac5af0174a04db5e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:10.719308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ced5d-2840-4ea6-96f2-28f8223369f4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:10.720752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ced5d-2840-4ea6-96f2-28f8223369f4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:10.722231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ced5d-2840-4ea6-96f2-28f8223369f4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:10.736952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 820/1650] 10.222.0.20 () {54 vars in 962 bytes} [Sat Jul 30 04:58:10 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:10.738681 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96e88c9f3e374d7d89f4d2a0a1efc3a5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.741322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96e88c9f3e374d7d89f4d2a0a1efc3a5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.742701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31415d08-0c50-416e-aa7b-97636ba45aed None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:10.743251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31415d08-0c50-416e-aa7b-97636ba45aed None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:10.745816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31415d08-0c50-416e-aa7b-97636ba45aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:10.747067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96e88c9f3e374d7d89f4d2a0a1efc3a5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.794828 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:10.801784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55975757-5f87-472b-acc9-8717d309ff28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96e88c9f3e374d7d89f4d2a0a1efc3a5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:10.815973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-31415d08-0c50-416e-aa7b-97636ba45aed None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:10.820961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-31415d08-0c50-416e-aa7b-97636ba45aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:10.854890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 831/1651] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:10 2022] DELETE /identity/v3/users/96e88c9f3e374d7d89f4d2a0a1efc3a5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:10.856693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099bb0c-bc45-49c8-a5ad-69f453b022bf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:10.857201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099bb0c-bc45-49c8-a5ad-69f453b022bf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:10.857832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099bb0c-bc45-49c8-a5ad-69f453b022bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:10.875988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:10.886504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2099bb0c-bc45-49c8-a5ad-69f453b022bf None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:10.888385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2099bb0c-bc45-49c8-a5ad-69f453b022bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:10.976602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 821/1652] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:10.978544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484aa2eb-3b8b-46ab-a83b-1f037c2c522a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:10.980171 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484aa2eb-3b8b-46ab-a83b-1f037c2c522a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:10.980555 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484aa2eb-3b8b-46ab-a83b-1f037c2c522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:11.037845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-484aa2eb-3b8b-46ab-a83b-1f037c2c522a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:11.039153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-484aa2eb-3b8b-46ab-a83b-1f037c2c522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:11.147236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 832/1653] 10.222.0.20 () {58 vars in 1059 bytes} [Sat Jul 30 04:58:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:11.149014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099bb0c-bc45-49c8-a5ad-69f453b022bf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:11.150582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2099bb0c-bc45-49c8-a5ad-69f453b022bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:11.182691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 822/1654] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:11.184194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d379e-7f6d-455d-8db2-179e0d42ad18 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:11.185998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d379e-7f6d-455d-8db2-179e0d42ad18 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:11.186281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d379e-7f6d-455d-8db2-179e0d42ad18 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:11.189578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 823/1655] 10.222.0.20 () {54 vars in 912 bytes} [Sat Jul 30 04:58:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:11.195909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b9bc-aef3-46fe-8765-c5e4a5746ebe None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:11.196229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b9bc-aef3-46fe-8765-c5e4a5746ebe None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:11.197400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b9bc-aef3-46fe-8765-c5e4a5746ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:11.242007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5934b9bc-aef3-46fe-8765-c5e4a5746ebe None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:11.246582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5934b9bc-aef3-46fe-8765-c5e4a5746ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:11.267557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 824/1656] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 04:58:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:11.270126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b9bc-aef3-46fe-8765-c5e4a5746ebe None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:11.272472 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5934b9bc-aef3-46fe-8765-c5e4a5746ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:11.281932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099bb0c-bc45-49c8-a5ad-69f453b022bf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:11.323355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b473-b61c-4d18-ac96-9eaf93150041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:11.323355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b473-b61c-4d18-ac96-9eaf93150041 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:11.323355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b473-b61c-4d18-ac96-9eaf93150041 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:11.323355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b473-b61c-4d18-ac96-9eaf93150041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:11.326377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b9bc-aef3-46fe-8765-c5e4a5746ebe None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:11.326377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a0eb5-8af8-45c2-abf2-19076e8a24e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:11.326377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a0eb5-8af8-45c2-abf2-19076e8a24e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:11.326377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a0eb5-8af8-45c2-abf2-19076e8a24e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:11.327006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a0eb5-8af8-45c2-abf2-19076e8a24e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:11.327142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030b473-b61c-4d18-ac96-9eaf93150041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:11.327142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030b473-b61c-4d18-ac96-9eaf93150041 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:11.339433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a0eb5-8af8-45c2-abf2-19076e8a24e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:11.341936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a0eb5-8af8-45c2-abf2-19076e8a24e6 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:11.351746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 833/1657] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:11.354908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b473-b61c-4d18-ac96-9eaf93150041 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:11.365278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a030b473-b61c-4d18-ac96-9eaf93150041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:11.378108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 825/1658] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:11.454122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b473-b61c-4d18-ac96-9eaf93150041 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:11.461262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cec5c-fa6c-4ec5-8350-98fe51ceb083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:11.461900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cec5c-fa6c-4ec5-8350-98fe51ceb083 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:11.462179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cec5c-fa6c-4ec5-8350-98fe51ceb083 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:11.463380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cec5c-fa6c-4ec5-8350-98fe51ceb083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:11.482373 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cec5c-fa6c-4ec5-8350-98fe51ceb083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:11.484404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cec5c-fa6c-4ec5-8350-98fe51ceb083 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:11.501163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 834/1659] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:12.156185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a0eb5-8af8-45c2-abf2-19076e8a24e6 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:12.158846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2a8a0eb5-8af8-45c2-abf2-19076e8a24e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:12.201745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a0eb5-8af8-45c2-abf2-19076e8a24e6 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:12.237104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:12.238037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:12.239120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:12.241407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] PATH_INFO: `/v3/projects/e479b62b596746669c1a6d8470c4faed` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:12.247537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e479b62b596746669c1a6d8470c4faed)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:12.250157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 30 04:58:12.250157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 30 04:58:12.250157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: Jul 30 04:58:12.255417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:12.385097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e479b62b596746669c1a6d8470c4faed', 'request_id': 'req-3a59229a-9cd8-4da3-9912-0b4a375f8a84'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:12.411084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e479b62b596746669c1a6d8470c4faed', 'request_id': 'req-3a59229a-9cd8-4da3-9912-0b4a375f8a84'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:12.421083 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e479b62b596746669c1a6d8470c4faed', 'request_id': 'req-3a59229a-9cd8-4da3-9912-0b4a375f8a84'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:12.472458 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] The token cache is being invalidate because project e479b62b596746669c1a6d8470c4faed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:12.473560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:12.475051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:12.484040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 826/1660] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:12 2022] DELETE /identity/v3/projects/e479b62b596746669c1a6d8470c4faed => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:12.487935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cec5c-fa6c-4ec5-8350-98fe51ceb083 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:12.510629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cec5c-fa6c-4ec5-8350-98fe51ceb083 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:12.723738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:12.726633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:12.747580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:12.780842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:12.790707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:12.813520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:12.814408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:12.816739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] PATH_INFO: `/v3/users/20afe06e885e451f8e7c79b1ceb6eee9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:12.820603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20afe06e885e451f8e7c79b1ceb6eee9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:12.849305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:12.960398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20afe06e885e451f8e7c79b1ceb6eee9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:12.969735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59229a-9cd8-4da3-9912-0b4a375f8a84 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:12.981174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20afe06e885e451f8e7c79b1ceb6eee9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:12.982328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20afe06e885e451f8e7c79b1ceb6eee9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:12.984171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20afe06e885e451f8e7c79b1ceb6eee9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:12.985777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:12.986618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:12.987004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:12.987615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] PATH_INFO: `/v3/projects/0a7dc50349e84acba8f9476de06cd75d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:12.990186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7dc50349e84acba8f9476de06cd75d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:12.996314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:13.031347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20afe06e885e451f8e7c79b1ceb6eee9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.070041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 835/1661] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:12 2022] DELETE /identity/v3/users/20afe06e885e451f8e7c79b1ceb6eee9 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:13.148356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7dc50349e84acba8f9476de06cd75d', 'request_id': 'req-9c83f4d3-a7fe-4bed-ad73-cf348882f051'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.162428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7dc50349e84acba8f9476de06cd75d', 'request_id': 'req-9c83f4d3-a7fe-4bed-ad73-cf348882f051'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.180633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7dc50349e84acba8f9476de06cd75d', 'request_id': 'req-9c83f4d3-a7fe-4bed-ad73-cf348882f051'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.205023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] The token cache is being invalidate because project 0a7dc50349e84acba8f9476de06cd75d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:13.205332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.210974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.214778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 827/1662] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:12 2022] DELETE /identity/v3/projects/0a7dc50349e84acba8f9476de06cd75d => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:13.218428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:13.223538 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:13.424714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f84e6-7e8c-478e-9693-180404a27bd6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:13.504919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:13.505558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:13.505928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:13.507910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] PATH_INFO: `/v3/users/b9392b17abed4984ad8d064507e243e8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:13.512503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9392b17abed4984ad8d064507e243e8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:13.516257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:13.541395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:13.637772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9392b17abed4984ad8d064507e243e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.649220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9392b17abed4984ad8d064507e243e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.650728 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9392b17abed4984ad8d064507e243e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.651360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9392b17abed4984ad8d064507e243e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.669469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b2919af-6df7-40cd-ab9c-d8b905388c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9392b17abed4984ad8d064507e243e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:13.690643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:13.692174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:13.698698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 836/1663] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:13 2022] DELETE /identity/v3/users/b9392b17abed4984ad8d064507e243e8 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:13.720004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83f4d3-a7fe-4bed-ad73-cf348882f051 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:13.913588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:13.914520 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:13.914960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:13.917890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] PATH_INFO: `/v3/users/ad49f82a91914d978f3cc15e1b7e0c29` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:13.929170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad49f82a91914d978f3cc15e1b7e0c29)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:13.936221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:14.060962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:14.064721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad49f82a91914d978f3cc15e1b7e0c29'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.098804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad49f82a91914d978f3cc15e1b7e0c29'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.101283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad49f82a91914d978f3cc15e1b7e0c29'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.102316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad49f82a91914d978f3cc15e1b7e0c29'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.173866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad49f82a91914d978f3cc15e1b7e0c29'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.218635 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 828/1664] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:13 2022] DELETE /identity/v3/users/ad49f82a91914d978f3cc15e1b7e0c29 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:14.223866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91871b-5c2f-4d23-a7c8-f466a2b6a562 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:14.224040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91871b-5c2f-4d23-a7c8-f466a2b6a562 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:14.224416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91871b-5c2f-4d23-a7c8-f466a2b6a562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:14.292755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9d91871b-5c2f-4d23-a7c8-f466a2b6a562 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:14.294202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9d91871b-5c2f-4d23-a7c8-f466a2b6a562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:14.515076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 837/1665] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:14.570997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:14.573625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:14.644156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91871b-5c2f-4d23-a7c8-f466a2b6a562 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:14.645715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9d91871b-5c2f-4d23-a7c8-f466a2b6a562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:14.723725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91871b-5c2f-4d23-a7c8-f466a2b6a562 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:14.745041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:14.748600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:14.748903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:14.750864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] PATH_INFO: `/v3/projects/c9e7f1507317483cb11d635432823ce6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:14.757491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e7f1507317483cb11d635432823ce6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:14.762616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:14.845471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f641b6-b345-4b74-9e08-c81144ed04d0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:14.859366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee9f57-1c25-412c-ac33-8ffce9e04e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:14.859998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee9f57-1c25-412c-ac33-8ffce9e04e53 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:14.860283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee9f57-1c25-412c-ac33-8ffce9e04e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:14.860560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee9f57-1c25-412c-ac33-8ffce9e04e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:14.881337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee9f57-1c25-412c-ac33-8ffce9e04e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:14.883090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee9f57-1c25-412c-ac33-8ffce9e04e53 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:14.906848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 829/1666] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:14.907654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e7f1507317483cb11d635432823ce6', 'request_id': 'req-2725af06-55eb-41f2-b4cf-b61107d79c7e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.927878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e7f1507317483cb11d635432823ce6', 'request_id': 'req-2725af06-55eb-41f2-b4cf-b61107d79c7e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.935905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e7f1507317483cb11d635432823ce6', 'request_id': 'req-2725af06-55eb-41f2-b4cf-b61107d79c7e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.954410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] The token cache is being invalidate because project c9e7f1507317483cb11d635432823ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:14.954733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.955422 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:14.966835 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 838/1667] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:14 2022] DELETE /identity/v3/projects/c9e7f1507317483cb11d635432823ce6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:58:14.968140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee9f57-1c25-412c-ac33-8ffce9e04e53 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:14.985716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee9f57-1c25-412c-ac33-8ffce9e04e53 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:15.129292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:15.130201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:15.130485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:15.130914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] PATH_INFO: `/v3/users/97f5de3b9a7641c09774c445344c2a1f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:15.133582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f5de3b9a7641c09774c445344c2a1f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:15.138166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:15.249012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f5de3b9a7641c09774c445344c2a1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.264195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f5de3b9a7641c09774c445344c2a1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.267825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f5de3b9a7641c09774c445344c2a1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.270946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f5de3b9a7641c09774c445344c2a1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.285415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:15.298722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f5de3b9a7641c09774c445344c2a1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.334260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 830/1668] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:14 2022] DELETE /identity/v3/users/97f5de3b9a7641c09774c445344c2a1f => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:15.581112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:15.583941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:15.611333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725af06-55eb-41f2-b4cf-b61107d79c7e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:15.628929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:15.745510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:15.746311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:15.746707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:15.746996 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] PATH_INFO: `/v3/projects/5719fd43ba514a39b397de32494363f8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:15.748331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5719fd43ba514a39b397de32494363f8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:15.750527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:15.778145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:15.779766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:15.805266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595f32e-301c-43fc-8751-09cceb806ffd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:15.812243 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:15.813487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:15.814018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:15.814560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] PATH_INFO: `/v3/projects/b5bc85452c4a4e37ad3b9c0800f05e46` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:15.816274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bc85452c4a4e37ad3b9c0800f05e46)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:15.820106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:15.844988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5719fd43ba514a39b397de32494363f8', 'request_id': 'req-f570045d-6371-4078-98fe-2ac435acac5e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.857122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5719fd43ba514a39b397de32494363f8', 'request_id': 'req-f570045d-6371-4078-98fe-2ac435acac5e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.864666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5719fd43ba514a39b397de32494363f8', 'request_id': 'req-f570045d-6371-4078-98fe-2ac435acac5e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.891045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] The token cache is being invalidate because project 5719fd43ba514a39b397de32494363f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:15.891946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.910261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f570045d-6371-4078-98fe-2ac435acac5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.914054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 839/1669] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:15 2022] DELETE /identity/v3/projects/5719fd43ba514a39b397de32494363f8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:15.919810 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff17d3-dd90-4d80-b360-e11feebaf375 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:15.921217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff17d3-dd90-4d80-b360-e11feebaf375 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:15.922831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff17d3-dd90-4d80-b360-e11feebaf375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:15.941221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bc85452c4a4e37ad3b9c0800f05e46', 'request_id': 'req-0561c3dc-9634-49e3-99d5-189fcd28bbee'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.967229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bc85452c4a4e37ad3b9c0800f05e46', 'request_id': 'req-0561c3dc-9634-49e3-99d5-189fcd28bbee'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.978380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bc85452c4a4e37ad3b9c0800f05e46', 'request_id': 'req-0561c3dc-9634-49e3-99d5-189fcd28bbee'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:15.983906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-65ff17d3-dd90-4d80-b360-e11feebaf375 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:15.987488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65ff17d3-dd90-4d80-b360-e11feebaf375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:16.027781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] The token cache is being invalidate because project b5bc85452c4a4e37ad3b9c0800f05e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:16.028214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:16.029102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0561c3dc-9634-49e3-99d5-189fcd28bbee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:16.049609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 831/1670] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:15 2022] DELETE /identity/v3/projects/b5bc85452c4a4e37ad3b9c0800f05e46 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:16.053915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab943b-a3f3-4be8-8ddc-4a112b0c442f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:16.054668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab943b-a3f3-4be8-8ddc-4a112b0c442f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:16.058140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab943b-a3f3-4be8-8ddc-4a112b0c442f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:16.105967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fbab943b-a3f3-4be8-8ddc-4a112b0c442f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:16.109264 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fbab943b-a3f3-4be8-8ddc-4a112b0c442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:16.176414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 840/1671] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:16.187348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d0178-2b5c-41c6-8faf-e7c398129de8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:16.187697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d0178-2b5c-41c6-8faf-e7c398129de8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:16.187912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d0178-2b5c-41c6-8faf-e7c398129de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:16.257164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1b7d0178-2b5c-41c6-8faf-e7c398129de8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:16.260205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1b7d0178-2b5c-41c6-8faf-e7c398129de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:16.289328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 841/1672] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:16.293181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d0178-2b5c-41c6-8faf-e7c398129de8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:16.316959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 832/1673] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:16.318490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10e648-e269-4efe-8f21-ebc503c3881c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:16.328531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10e648-e269-4efe-8f21-ebc503c3881c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:16.328807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10e648-e269-4efe-8f21-ebc503c3881c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:16.334914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d0178-2b5c-41c6-8faf-e7c398129de8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:16.361598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ea975-3209-4f36-925b-ab6a4fa32be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:16.362262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ea975-3209-4f36-925b-ab6a4fa32be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:16.362631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ea975-3209-4f36-925b-ab6a4fa32be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:16.362908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ea975-3209-4f36-925b-ab6a4fa32be5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:16.368915 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ea975-3209-4f36-925b-ab6a4fa32be5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:16.369579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ea975-3209-4f36-925b-ab6a4fa32be5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:16.376708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ea975-3209-4f36-925b-ab6a4fa32be5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:16.437634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 842/1674] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:16.437838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0e10e648-e269-4efe-8f21-ebc503c3881c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:16.437838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0e10e648-e269-4efe-8f21-ebc503c3881c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:16.439661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ea975-3209-4f36-925b-ab6a4fa32be5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:16.456398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ea975-3209-4f36-925b-ab6a4fa32be5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:16.473048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 833/1675] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:16.473970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10e648-e269-4efe-8f21-ebc503c3881c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:16.475186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe1893-ff96-4825-8bf4-86029a640562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:16.475834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1893-ff96-4825-8bf4-86029a640562 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:16.476114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1893-ff96-4825-8bf4-86029a640562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:16.476382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1893-ff96-4825-8bf4-86029a640562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:16.477052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe1893-ff96-4825-8bf4-86029a640562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:16.483358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe1893-ff96-4825-8bf4-86029a640562 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:16.496874 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10e648-e269-4efe-8f21-ebc503c3881c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:16.516358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5576fec-1c9f-4011-96e1-2fd366e3f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:16.517008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5576fec-1c9f-4011-96e1-2fd366e3f75a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:16.517285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5576fec-1c9f-4011-96e1-2fd366e3f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:16.517556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5576fec-1c9f-4011-96e1-2fd366e3f75a admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:16.518258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5576fec-1c9f-4011-96e1-2fd366e3f75a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:16.518587 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5576fec-1c9f-4011-96e1-2fd366e3f75a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:16.520200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5576fec-1c9f-4011-96e1-2fd366e3f75a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:16.566495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 843/1676] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:16 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:16.574196 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe1893-ff96-4825-8bf4-86029a640562 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:16.579019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 834/1677] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:16.580104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5576fec-1c9f-4011-96e1-2fd366e3f75a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:16.585191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:16.621283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5576fec-1c9f-4011-96e1-2fd366e3f75a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:16.736047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:16.770978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414ea53-7908-4d46-a8cc-839deb83dd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:16.771627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414ea53-7908-4d46-a8cc-839deb83dd8a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:16.771902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414ea53-7908-4d46-a8cc-839deb83dd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:16.772174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414ea53-7908-4d46-a8cc-839deb83dd8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:16.772857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414ea53-7908-4d46-a8cc-839deb83dd8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:16.775046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414ea53-7908-4d46-a8cc-839deb83dd8a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:16.783298 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe1893-ff96-4825-8bf4-86029a640562 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:16.788935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add66b8-b39f-45b7-b4be-56d5c4859bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:16.790843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add66b8-b39f-45b7-b4be-56d5c4859bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:16.790843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add66b8-b39f-45b7-b4be-56d5c4859bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:16.791678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add66b8-b39f-45b7-b4be-56d5c4859bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:16.792702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add66b8-b39f-45b7-b4be-56d5c4859bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:16.796555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add66b8-b39f-45b7-b4be-56d5c4859bf0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:16.843584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 844/1678] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:16 2022] POST /identity/v3/users => generated 284 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:16.847837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add66b8-b39f-45b7-b4be-56d5c4859bf0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:16.853951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 835/1679] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:16 2022] POST /identity/v3/projects => generated 371 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:16.858454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414ea53-7908-4d46-a8cc-839deb83dd8a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:16.865529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add66b8-b39f-45b7-b4be-56d5c4859bf0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:16.956829 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5fbe2-6efa-4a9d-86c3-42409546c71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:16.958239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5fbe2-6efa-4a9d-86c3-42409546c71a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:16.959401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5fbe2-6efa-4a9d-86c3-42409546c71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:16.960359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5fbe2-6efa-4a9d-86c3-42409546c71a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:16.961233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5fbe2-6efa-4a9d-86c3-42409546c71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:16.964389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5fbe2-6efa-4a9d-86c3-42409546c71a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:16.983394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 845/1680] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:16.984647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5fbe2-6efa-4a9d-86c3-42409546c71a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:17.004017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5fbe2-6efa-4a9d-86c3-42409546c71a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:17.023382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414ea53-7908-4d46-a8cc-839deb83dd8a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:17.029653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd980108-8c19-4e5e-be63-c861fb29973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:17.030645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd980108-8c19-4e5e-be63-c861fb29973b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.031037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd980108-8c19-4e5e-be63-c861fb29973b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.031524 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd980108-8c19-4e5e-be63-c861fb29973b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.032361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd980108-8c19-4e5e-be63-c861fb29973b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:17.043982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd980108-8c19-4e5e-be63-c861fb29973b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:17.045124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947c24d-7ae5-4fef-98be-be25c2f1baef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:17.046079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c24d-7ae5-4fef-98be-be25c2f1baef admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.046489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c24d-7ae5-4fef-98be-be25c2f1baef admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.046728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c24d-7ae5-4fef-98be-be25c2f1baef admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.047399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947c24d-7ae5-4fef-98be-be25c2f1baef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:17.056574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947c24d-7ae5-4fef-98be-be25c2f1baef admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:17.070537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 846/1681] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:17.074034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd980108-8c19-4e5e-be63-c861fb29973b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:17.116577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd980108-8c19-4e5e-be63-c861fb29973b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:17.119055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 836/1682] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:16 2022] POST /identity/v3/users => generated 286 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:17.122634 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947c24d-7ae5-4fef-98be-be25c2f1baef admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:17.133555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf962bc-7cd0-4e5f-9ef3-31948fa81554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:17.135629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf962bc-7cd0-4e5f-9ef3-31948fa81554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.136117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf962bc-7cd0-4e5f-9ef3-31948fa81554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.137555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf962bc-7cd0-4e5f-9ef3-31948fa81554 admin admin] PATH_INFO: `/v3/projects/e96985506fe649b694f9b87979900cd9/users/0a24c464748a4d2882eba3536855f9e3/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.139476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947c24d-7ae5-4fef-98be-be25c2f1baef admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:17.155181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a2d7c-95bc-47db-bc6e-840eb6656a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:17.156091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a2d7c-95bc-47db-bc6e-840eb6656a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.156509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a2d7c-95bc-47db-bc6e-840eb6656a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.156911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a2d7c-95bc-47db-bc6e-840eb6656a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.157754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a2d7c-95bc-47db-bc6e-840eb6656a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:17.159611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a2d7c-95bc-47db-bc6e-840eb6656a3b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:17.185613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf962bc-7cd0-4e5f-9ef3-31948fa81554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96985506fe649b694f9b87979900cd9, user_id=0a24c464748a4d2882eba3536855f9e3, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:17.189085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf962bc-7cd0-4e5f-9ef3-31948fa81554 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:17.189373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 837/1683] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:58:17.190347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a2d7c-95bc-47db-bc6e-840eb6656a3b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:17.195964 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f86a2d7c-95bc-47db-bc6e-840eb6656a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:17.215613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a2d7c-95bc-47db-bc6e-840eb6656a3b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:17.224531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 847/1684] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:17 2022] PUT /identity/v3/projects/e96985506fe649b694f9b87979900cd9/users/0a24c464748a4d2882eba3536855f9e3/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:17.224738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:17.225631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf962bc-7cd0-4e5f-9ef3-31948fa81554 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:17.236104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.236457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.237836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] PATH_INFO: `/v3/projects/54d694b49b424dcbaa8349d6605f5e34` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.239370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=54d694b49b424dcbaa8349d6605f5e34)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:17.241763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:17.312052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54d694b49b424dcbaa8349d6605f5e34', 'request_id': 'req-5141ec33-3b96-4147-b4c2-257e23223f2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.324057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54d694b49b424dcbaa8349d6605f5e34', 'request_id': 'req-5141ec33-3b96-4147-b4c2-257e23223f2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.335236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54d694b49b424dcbaa8349d6605f5e34', 'request_id': 'req-5141ec33-3b96-4147-b4c2-257e23223f2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.343086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:17.362672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf962bc-7cd0-4e5f-9ef3-31948fa81554 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:17.385747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] The token cache is being invalidate because project 54d694b49b424dcbaa8349d6605f5e34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:17.386845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.387729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5141ec33-3b96-4147-b4c2-257e23223f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.390243 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 838/1685] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:17 2022] DELETE /identity/v3/projects/54d694b49b424dcbaa8349d6605f5e34 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:17.392555 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627f2e-7a6b-46ee-872f-bb519b710fe5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.392988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627f2e-7a6b-46ee-872f-bb519b710fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.393395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627f2e-7a6b-46ee-872f-bb519b710fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.445825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-22627f2e-7a6b-46ee-872f-bb519b710fe5 None None] MFA Rules not processed for user `0a24c464748a4d2882eba3536855f9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:17.450155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-22627f2e-7a6b-46ee-872f-bb519b710fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:17.493420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:17.494690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba9fbf-13db-4ccf-99de-0387fe7b1278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:17.496050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba9fbf-13db-4ccf-99de-0387fe7b1278 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.496050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba9fbf-13db-4ccf-99de-0387fe7b1278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.497742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba9fbf-13db-4ccf-99de-0387fe7b1278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.500851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba9fbf-13db-4ccf-99de-0387fe7b1278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:17.502617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba9fbf-13db-4ccf-99de-0387fe7b1278 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:17.526635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 848/1686] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:17.528878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba9fbf-13db-4ccf-99de-0387fe7b1278 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:17.532924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74ba9fbf-13db-4ccf-99de-0387fe7b1278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:17.576826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba9fbf-13db-4ccf-99de-0387fe7b1278 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:17.578916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 839/1687] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:17.580385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a6509-cbc0-4dd5-92fc-5151c61d8045 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.580735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a6509-cbc0-4dd5-92fc-5151c61d8045 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.581018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a6509-cbc0-4dd5-92fc-5151c61d8045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.592471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:17.593139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.593421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.593693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] PATH_INFO: `/v3/projects/922209a39e114d2dbd25c771d269631e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.597556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=922209a39e114d2dbd25c771d269631e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:17.599777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:17.610357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1e9a6509-cbc0-4dd5-92fc-5151c61d8045 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:17.612071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1e9a6509-cbc0-4dd5-92fc-5151c61d8045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:17.621068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 840/1688] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:17.622330 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a6509-cbc0-4dd5-92fc-5151c61d8045 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:17.623894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1e9a6509-cbc0-4dd5-92fc-5151c61d8045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:17.639068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a6509-cbc0-4dd5-92fc-5151c61d8045 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:17.650977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8122719-d3a0-4f22-a940-82eb1a972d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:17.652035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8122719-d3a0-4f22-a940-82eb1a972d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.652620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8122719-d3a0-4f22-a940-82eb1a972d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.653043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8122719-d3a0-4f22-a940-82eb1a972d7d admin admin] PATH_INFO: `/v3/projects/0ecccdebfe9e4cf29cc1f6d2b21f7247/users/36df16ba568648bf985c4782476672e7/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.691177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8122719-d3a0-4f22-a940-82eb1a972d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ecccdebfe9e4cf29cc1f6d2b21f7247, user_id=36df16ba568648bf985c4782476672e7, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:17.706721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8122719-d3a0-4f22-a940-82eb1a972d7d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:17.729138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922209a39e114d2dbd25c771d269631e', 'request_id': 'req-9365f34d-ce38-478b-9d5f-502d493d8049'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.736110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 841/1689] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:17 2022] PUT /identity/v3/projects/0ecccdebfe9e4cf29cc1f6d2b21f7247/users/36df16ba568648bf985c4782476672e7/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:17.737593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8122719-d3a0-4f22-a940-82eb1a972d7d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:17.740136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a8122719-d3a0-4f22-a940-82eb1a972d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:17.749191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922209a39e114d2dbd25c771d269631e', 'request_id': 'req-9365f34d-ce38-478b-9d5f-502d493d8049'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.761277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922209a39e114d2dbd25c771d269631e', 'request_id': 'req-9365f34d-ce38-478b-9d5f-502d493d8049'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.789411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] The token cache is being invalidate because project 922209a39e114d2dbd25c771d269631e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:17.789719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.798051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9365f34d-ce38-478b-9d5f-502d493d8049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:17.800513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 849/1690] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:17 2022] DELETE /identity/v3/projects/922209a39e114d2dbd25c771d269631e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:17.802069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1d9f-bc2f-4073-be1f-0ce56e5b56ae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.802417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1d9f-bc2f-4073-be1f-0ce56e5b56ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.802769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1d9f-bc2f-4073-be1f-0ce56e5b56ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.853321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-780b1d9f-bc2f-4073-be1f-0ce56e5b56ae None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:17.854851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-780b1d9f-bc2f-4073-be1f-0ce56e5b56ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:17.887228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8122719-d3a0-4f22-a940-82eb1a972d7d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:17.895304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d91ad6-f22d-4c5f-b5d5-880a28f15f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:17.896460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d91ad6-f22d-4c5f-b5d5-880a28f15f8e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.896745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d91ad6-f22d-4c5f-b5d5-880a28f15f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.897025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d91ad6-f22d-4c5f-b5d5-880a28f15f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.897709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d91ad6-f22d-4c5f-b5d5-880a28f15f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:17.900679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d91ad6-f22d-4c5f-b5d5-880a28f15f8e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:17.944810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 842/1691] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:17 2022] POST /identity/v3/projects => generated 369 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:17.947431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04af8-c43b-4a99-892b-82e412a7cab4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.947943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04af8-c43b-4a99-892b-82e412a7cab4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.948224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04af8-c43b-4a99-892b-82e412a7cab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:17.985006 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 850/1692] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:17.988293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90e95a-cef9-477d-a058-c65aaa6d1cab None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:17.988293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90e95a-cef9-477d-a058-c65aaa6d1cab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:17.988293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90e95a-cef9-477d-a058-c65aaa6d1cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.000690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c5a04af8-c43b-4a99-892b-82e412a7cab4 None None] MFA Rules not processed for user `36df16ba568648bf985c4782476672e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:18.002353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c5a04af8-c43b-4a99-892b-82e412a7cab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:18.034517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ba90e95a-cef9-477d-a058-c65aaa6d1cab None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:18.036386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba90e95a-cef9-477d-a058-c65aaa6d1cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:18.065823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:18.145669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 843/1693] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:18.153168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04af8-c43b-4a99-892b-82e412a7cab4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.156375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c5a04af8-c43b-4a99-892b-82e412a7cab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:18.160796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 851/1694] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:18.163717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90e95a-cef9-477d-a058-c65aaa6d1cab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.182091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90e95a-cef9-477d-a058-c65aaa6d1cab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:18.185177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04af8-c43b-4a99-892b-82e412a7cab4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:18.194503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:18.197255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:18.197708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:18.198288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.198743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1a978-13fc-4f50-90a3-770989dfd0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:18.199590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1a978-13fc-4f50-90a3-770989dfd0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:18.200664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:18.201185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1a978-13fc-4f50-90a3-770989dfd0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:18.201642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:18.201896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1a978-13fc-4f50-90a3-770989dfd0a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.203347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1a978-13fc-4f50-90a3-770989dfd0a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:18.205801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:18.208469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1a978-13fc-4f50-90a3-770989dfd0a6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:18.231509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:18.233407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 852/1695] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:18.235600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.240098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:18.258117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 844/1696] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:18 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:18.264244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68696415-ea58-447f-8953-afe1627f02f5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:18.264459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68696415-ea58-447f-8953-afe1627f02f5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:18.265433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68696415-ea58-447f-8953-afe1627f02f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.277155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bb03d-b6de-4a6f-8434-8d9a18c981bf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:18.288921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c438b-ad01-4a17-b3ce-88a96073c908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:18.289980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c438b-ad01-4a17-b3ce-88a96073c908 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:18.291799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c438b-ad01-4a17-b3ce-88a96073c908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:18.292503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c438b-ad01-4a17-b3ce-88a96073c908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.294425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c438b-ad01-4a17-b3ce-88a96073c908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:18.297404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c438b-ad01-4a17-b3ce-88a96073c908 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:18.311559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-68696415-ea58-447f-8953-afe1627f02f5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:18.313153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-68696415-ea58-447f-8953-afe1627f02f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:18.339318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 845/1697] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:18.340749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68696415-ea58-447f-8953-afe1627f02f5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.361566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 853/1698] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:18 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:18.368023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c438b-ad01-4a17-b3ce-88a96073c908 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.371639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68696415-ea58-447f-8953-afe1627f02f5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:18.527306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb2f3f-5226-46a9-8c01-9a46c7ea0507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:18.528188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb2f3f-5226-46a9-8c01-9a46c7ea0507 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:18.528830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb2f3f-5226-46a9-8c01-9a46c7ea0507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:18.532941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb2f3f-5226-46a9-8c01-9a46c7ea0507 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.532941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb2f3f-5226-46a9-8c01-9a46c7ea0507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:18.534928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb2f3f-5226-46a9-8c01-9a46c7ea0507 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:18.545143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c438b-ad01-4a17-b3ce-88a96073c908 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:18.557189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c960480-9bf1-4ac2-b04a-d2afa8ab684f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:18.558059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c960480-9bf1-4ac2-b04a-d2afa8ab684f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:18.558271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c960480-9bf1-4ac2-b04a-d2afa8ab684f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:18.558577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c960480-9bf1-4ac2-b04a-d2afa8ab684f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.559340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c960480-9bf1-4ac2-b04a-d2afa8ab684f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:18.563293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c960480-9bf1-4ac2-b04a-d2afa8ab684f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:18.598484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 854/1699] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:18.601641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c960480-9bf1-4ac2-b04a-d2afa8ab684f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.606479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 846/1700] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:18 2022] POST /identity/v3/projects => generated 375 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:18.607461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb2f3f-5226-46a9-8c01-9a46c7ea0507 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.625884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c960480-9bf1-4ac2-b04a-d2afa8ab684f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:18.692602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb2f3f-5226-46a9-8c01-9a46c7ea0507 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:18.700481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bafcb-bb20-491d-a6a0-2172eb371e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:18.701344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bafcb-bb20-491d-a6a0-2172eb371e42 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:18.701768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bafcb-bb20-491d-a6a0-2172eb371e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:18.702207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bafcb-bb20-491d-a6a0-2172eb371e42 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.703447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bafcb-bb20-491d-a6a0-2172eb371e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:18.705844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bafcb-bb20-491d-a6a0-2172eb371e42 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:18.740791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 847/1701] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:18 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:18.741976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bafcb-bb20-491d-a6a0-2172eb371e42 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.762019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bafcb-bb20-491d-a6a0-2172eb371e42 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:18.762987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac208d-830a-4a62-92fa-911ccd1b4d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:18.764877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac208d-830a-4a62-92fa-911ccd1b4d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:18.766295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac208d-830a-4a62-92fa-911ccd1b4d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:18.766916 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac208d-830a-4a62-92fa-911ccd1b4d95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.768027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac208d-830a-4a62-92fa-911ccd1b4d95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:18.768527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac208d-830a-4a62-92fa-911ccd1b4d95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:18.769731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6217e3-a62d-4daa-8882-ee21277a4146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:18.770496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6217e3-a62d-4daa-8882-ee21277a4146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:18.770783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6217e3-a62d-4daa-8882-ee21277a4146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:18.771060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6217e3-a62d-4daa-8882-ee21277a4146 admin admin] PATH_INFO: `/v3/projects/361c2261b9134891b6aac5cd2762ea01/users/36aa2484e60f4de0934f2aa1667e12df/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:18.771539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac208d-830a-4a62-92fa-911ccd1b4d95 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:18.797665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6217e3-a62d-4daa-8882-ee21277a4146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=361c2261b9134891b6aac5cd2762ea01, user_id=36aa2484e60f4de0934f2aa1667e12df, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:18.801616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6217e3-a62d-4daa-8882-ee21277a4146 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:18.810557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 855/1702] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:18.811724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac208d-830a-4a62-92fa-911ccd1b4d95 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.825835 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac208d-830a-4a62-92fa-911ccd1b4d95 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:18.835061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 848/1703] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:18 2022] PUT /identity/v3/projects/361c2261b9134891b6aac5cd2762ea01/users/36aa2484e60f4de0934f2aa1667e12df/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:18.836616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6217e3-a62d-4daa-8882-ee21277a4146 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:18.860561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:18.995753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:19.012932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5acc8-ab34-4729-b9dc-a81f33079263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.013593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5acc8-ab34-4729-b9dc-a81f33079263 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.013881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5acc8-ab34-4729-b9dc-a81f33079263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.014158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5acc8-ab34-4729-b9dc-a81f33079263 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.014943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5acc8-ab34-4729-b9dc-a81f33079263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.016860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5acc8-ab34-4729-b9dc-a81f33079263 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.076297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 856/1704] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:18 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:19.082531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5acc8-ab34-4729-b9dc-a81f33079263 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.100185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6217e3-a62d-4daa-8882-ee21277a4146 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.113690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5acc8-ab34-4729-b9dc-a81f33079263 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.115436 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e70d26-b8d3-4041-aa9c-2afe2d75cb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.116261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e70d26-b8d3-4041-aa9c-2afe2d75cb61 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.123101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e70d26-b8d3-4041-aa9c-2afe2d75cb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.123599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e70d26-b8d3-4041-aa9c-2afe2d75cb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.124694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e70d26-b8d3-4041-aa9c-2afe2d75cb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.126723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e70d26-b8d3-4041-aa9c-2afe2d75cb61 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.138479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506597a-6e46-4eb1-b930-754fe53ae4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.139264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506597a-6e46-4eb1-b930-754fe53ae4ea admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.139552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506597a-6e46-4eb1-b930-754fe53ae4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.139831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506597a-6e46-4eb1-b930-754fe53ae4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.140519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506597a-6e46-4eb1-b930-754fe53ae4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.142689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506597a-6e46-4eb1-b930-754fe53ae4ea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.167432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 849/1705] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:19.171227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e70d26-b8d3-4041-aa9c-2afe2d75cb61 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.196173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e70d26-b8d3-4041-aa9c-2afe2d75cb61 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.210012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc553a7-600a-40f5-897c-a60767aa4b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.210292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc553a7-600a-40f5-897c-a60767aa4b30 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.210606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc553a7-600a-40f5-897c-a60767aa4b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.210874 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc553a7-600a-40f5-897c-a60767aa4b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.211563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc553a7-600a-40f5-897c-a60767aa4b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.213871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc553a7-600a-40f5-897c-a60767aa4b30 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.238659 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 857/1706] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:19 2022] POST /identity/v3/projects => generated 405 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:19.239907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506597a-6e46-4eb1-b930-754fe53ae4ea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.268244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 850/1707] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:19.279090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc553a7-600a-40f5-897c-a60767aa4b30 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.310012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc553a7-600a-40f5-897c-a60767aa4b30 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.422425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506597a-6e46-4eb1-b930-754fe53ae4ea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.437608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4429a4-c86e-4aea-8205-96df1a5c04b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.438194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1858dd-5422-42b1-85a5-c7efd5f59f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.438385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4429a4-c86e-4aea-8205-96df1a5c04b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.438735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4429a4-c86e-4aea-8205-96df1a5c04b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.439013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4429a4-c86e-4aea-8205-96df1a5c04b1 admin admin] PATH_INFO: `/v3/projects/1c38378b250a4014b700eb2863c96df7/users/87cc4f7a4f2647cfb08e742b9db00cec/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.439862 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1858dd-5422-42b1-85a5-c7efd5f59f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.440340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1858dd-5422-42b1-85a5-c7efd5f59f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.440806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1858dd-5422-42b1-85a5-c7efd5f59f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.441733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1858dd-5422-42b1-85a5-c7efd5f59f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.443614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1858dd-5422-42b1-85a5-c7efd5f59f9b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.467594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 858/1708] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:19.471716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1858dd-5422-42b1-85a5-c7efd5f59f9b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.475070 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4429a4-c86e-4aea-8205-96df1a5c04b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c38378b250a4014b700eb2863c96df7, user_id=87cc4f7a4f2647cfb08e742b9db00cec, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.484566 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4429a4-c86e-4aea-8205-96df1a5c04b1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.498695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1858dd-5422-42b1-85a5-c7efd5f59f9b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.509425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1aa665-307f-4688-ba78-85847f199070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.510391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1aa665-307f-4688-ba78-85847f199070 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.510951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1aa665-307f-4688-ba78-85847f199070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.511470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1aa665-307f-4688-ba78-85847f199070 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.512362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1aa665-307f-4688-ba78-85847f199070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.514755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1aa665-307f-4688-ba78-85847f199070 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.518405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 851/1709] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:19 2022] PUT /identity/v3/projects/1c38378b250a4014b700eb2863c96df7/users/87cc4f7a4f2647cfb08e742b9db00cec/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:19.519215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4429a4-c86e-4aea-8205-96df1a5c04b1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.584592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 859/1710] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:19 2022] POST /identity/v3/users => generated 303 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:19.585588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1aa665-307f-4688-ba78-85847f199070 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.608111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1aa665-307f-4688-ba78-85847f199070 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.655948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4429a4-c86e-4aea-8205-96df1a5c04b1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.673811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed113321-4dbf-4dd4-96ce-8b3167a40bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.674392 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed113321-4dbf-4dd4-96ce-8b3167a40bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.674701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed113321-4dbf-4dd4-96ce-8b3167a40bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.674982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed113321-4dbf-4dd4-96ce-8b3167a40bd9 admin admin] PATH_INFO: `/v3/projects/361c2261b9134891b6aac5cd2762ea01/users/36aa2484e60f4de0934f2aa1667e12df/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.679217 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed113321-4dbf-4dd4-96ce-8b3167a40bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=361c2261b9134891b6aac5cd2762ea01, user_id=36aa2484e60f4de0934f2aa1667e12df, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.683268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed113321-4dbf-4dd4-96ce-8b3167a40bd9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.722381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 852/1711] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:19 2022] PUT /identity/v3/projects/361c2261b9134891b6aac5cd2762ea01/users/36aa2484e60f4de0934f2aa1667e12df/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:19.723466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed113321-4dbf-4dd4-96ce-8b3167a40bd9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.764801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f515244-a11c-4d56-8c3d-feb1bf3adf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.765459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f515244-a11c-4d56-8c3d-feb1bf3adf88 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.765745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f515244-a11c-4d56-8c3d-feb1bf3adf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.766514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f515244-a11c-4d56-8c3d-feb1bf3adf88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.767213 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f515244-a11c-4d56-8c3d-feb1bf3adf88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.768926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f515244-a11c-4d56-8c3d-feb1bf3adf88 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.779988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:19.792908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 860/1712] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:19.793936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f515244-a11c-4d56-8c3d-feb1bf3adf88 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.805183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f515244-a11c-4d56-8c3d-feb1bf3adf88 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.826163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:19.883740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed113321-4dbf-4dd4-96ce-8b3167a40bd9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.897621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c70ca4-a9a2-4d10-8d89-20a1e710b3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.898270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c70ca4-a9a2-4d10-8d89-20a1e710b3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.898585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c70ca4-a9a2-4d10-8d89-20a1e710b3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.898861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c70ca4-a9a2-4d10-8d89-20a1e710b3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.900324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c70ca4-a9a2-4d10-8d89-20a1e710b3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.902553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c70ca4-a9a2-4d10-8d89-20a1e710b3f2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.906638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974ed1c-b8e1-4cdf-b183-2c08b7153475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.907794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974ed1c-b8e1-4cdf-b183-2c08b7153475 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.908086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974ed1c-b8e1-4cdf-b183-2c08b7153475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.908361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974ed1c-b8e1-4cdf-b183-2c08b7153475 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.909081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974ed1c-b8e1-4cdf-b183-2c08b7153475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:19.910730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974ed1c-b8e1-4cdf-b183-2c08b7153475 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:19.923044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 861/1713] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:19.924370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f29da2-f73f-4645-88f2-6fb2b9bf53f2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.924670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f29da2-f73f-4645-88f2-6fb2b9bf53f2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.924928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f29da2-f73f-4645-88f2-6fb2b9bf53f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.936856 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 853/1714] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:19.937653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974ed1c-b8e1-4cdf-b183-2c08b7153475 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:19.958374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974ed1c-b8e1-4cdf-b183-2c08b7153475 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:19.972371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-97f29da2-f73f-4645-88f2-6fb2b9bf53f2 None None] MFA Rules not processed for user `36aa2484e60f4de0934f2aa1667e12df`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:19.975397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee974471-fd61-43f3-a7b4-e71a4b8d9356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:19.976104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee974471-fd61-43f3-a7b4-e71a4b8d9356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:19.976461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee974471-fd61-43f3-a7b4-e71a4b8d9356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:19.976741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee974471-fd61-43f3-a7b4-e71a4b8d9356 admin admin] PATH_INFO: `/v3/projects/897900152f9e4d8090677c6da2474a32/users/bc4be97e0f274b54a29ba6d9df302ede/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:19.977263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97f29da2-f73f-4645-88f2-6fb2b9bf53f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:20.032560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee974471-fd61-43f3-a7b4-e71a4b8d9356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897900152f9e4d8090677c6da2474a32, user_id=bc4be97e0f274b54a29ba6d9df302ede, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:20.035602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee974471-fd61-43f3-a7b4-e71a4b8d9356 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:20.074820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 854/1715] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:19 2022] PUT /identity/v3/projects/897900152f9e4d8090677c6da2474a32/users/bc4be97e0f274b54a29ba6d9df302ede/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:20.075430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee974471-fd61-43f3-a7b4-e71a4b8d9356 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:20.167117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 862/1716] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:19 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:20.168047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f29da2-f73f-4645-88f2-6fb2b9bf53f2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:20.197321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f29da2-f73f-4645-88f2-6fb2b9bf53f2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:20.223150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b3a49-867d-4071-b318-483a7184541a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:20.223860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b3a49-867d-4071-b318-483a7184541a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:20.224148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b3a49-867d-4071-b318-483a7184541a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:20.224423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b3a49-867d-4071-b318-483a7184541a admin admin] PATH_INFO: `/v3/projects/1c38378b250a4014b700eb2863c96df7/users/87cc4f7a4f2647cfb08e742b9db00cec/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:20.237537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b3a49-867d-4071-b318-483a7184541a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c38378b250a4014b700eb2863c96df7, user_id=87cc4f7a4f2647cfb08e742b9db00cec, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:20.240309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b3a49-867d-4071-b318-483a7184541a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:20.242731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee974471-fd61-43f3-a7b4-e71a4b8d9356 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:20.266106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54e97e-b5aa-46db-96bf-d9e0f82628d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:20.266869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54e97e-b5aa-46db-96bf-d9e0f82628d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:20.267181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54e97e-b5aa-46db-96bf-d9e0f82628d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:20.267413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54e97e-b5aa-46db-96bf-d9e0f82628d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:20.269359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54e97e-b5aa-46db-96bf-d9e0f82628d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:20.277296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54e97e-b5aa-46db-96bf-d9e0f82628d0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:20.279754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 863/1717] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:20 2022] PUT /identity/v3/projects/1c38378b250a4014b700eb2863c96df7/users/87cc4f7a4f2647cfb08e742b9db00cec/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:20.281063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4763898-1a0c-4de4-b7a6-213ca03c84c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:20.281358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4763898-1a0c-4de4-b7a6-213ca03c84c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:20.282841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4763898-1a0c-4de4-b7a6-213ca03c84c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:20.335454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 855/1718] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:20.340544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3dd74-ad29-4905-8173-258379b50979 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:20.341960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3dd74-ad29-4905-8173-258379b50979 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:20.342795 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3dd74-ad29-4905-8173-258379b50979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:20.373833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c4763898-1a0c-4de4-b7a6-213ca03c84c0 None None] MFA Rules not processed for user `d3a56c6d44b74429a6616cdfc2869b71`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:20.378195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4763898-1a0c-4de4-b7a6-213ca03c84c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:20.447957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-76f3dd74-ad29-4905-8173-258379b50979 None None] MFA Rules not processed for user `bc4be97e0f274b54a29ba6d9df302ede`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:20.452524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-76f3dd74-ad29-4905-8173-258379b50979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:20.535151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:20.572794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 864/1719] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:20.574273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6b659-5965-4741-bd5c-72c1b4834701 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:20.574593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6b659-5965-4741-bd5c-72c1b4834701 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:20.574879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6b659-5965-4741-bd5c-72c1b4834701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:20.667612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ffd6b659-5965-4741-bd5c-72c1b4834701 None None] MFA Rules not processed for user `a8abb58f9c274e568d2d0fe3d4df11de`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:20.672724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ffd6b659-5965-4741-bd5c-72c1b4834701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:20.691648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 856/1720] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:20.692559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3dd74-ad29-4905-8173-258379b50979 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:20.694271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-76f3dd74-ad29-4905-8173-258379b50979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:20.776338 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:20.829106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3dd74-ad29-4905-8173-258379b50979 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:20.836116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd88d39-52e5-4103-b2d6-03957fe78791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:20.837695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd88d39-52e5-4103-b2d6-03957fe78791 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:20.838056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd88d39-52e5-4103-b2d6-03957fe78791 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:20.838429 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd88d39-52e5-4103-b2d6-03957fe78791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:20.847405 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 865/1721] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:20.860365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd925c-7ddf-440a-920f-8ed651b73c22 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:20.861016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd925c-7ddf-440a-920f-8ed651b73c22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:20.861732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd925c-7ddf-440a-920f-8ed651b73c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:20.899925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0cbd925c-7ddf-440a-920f-8ed651b73c22 None None] MFA Rules not processed for user `87cc4f7a4f2647cfb08e742b9db00cec`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:20.901481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0cbd925c-7ddf-440a-920f-8ed651b73c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:21.003361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd88d39-52e5-4103-b2d6-03957fe78791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:21.004850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd88d39-52e5-4103-b2d6-03957fe78791 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:21.030426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 857/1722] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:20 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:21.031302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd88d39-52e5-4103-b2d6-03957fe78791 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:21.126638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 866/1723] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:21.135441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd925c-7ddf-440a-920f-8ed651b73c22 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:21.143386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0cbd925c-7ddf-440a-920f-8ed651b73c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:21.232295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd88d39-52e5-4103-b2d6-03957fe78791 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:21.256757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8813ec-bcea-43e9-91fb-848ddc1346c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:21.257334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8813ec-bcea-43e9-91fb-848ddc1346c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:21.257592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8813ec-bcea-43e9-91fb-848ddc1346c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:21.257912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8813ec-bcea-43e9-91fb-848ddc1346c2 admin admin] PATH_INFO: `/v3/projects/50232a98be954d9f9a2e1564566733c5/users/7fa5e3cc28ca46d2a4f3b44f23c5a76b/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:21.291817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd925c-7ddf-440a-920f-8ed651b73c22 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:21.302310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf57ce-31a6-425e-ae43-3894c1e2d7a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:21.303153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf57ce-31a6-425e-ae43-3894c1e2d7a7 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:21.303542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf57ce-31a6-425e-ae43-3894c1e2d7a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:21.303924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf57ce-31a6-425e-ae43-3894c1e2d7a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:21.307992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8813ec-bcea-43e9-91fb-848ddc1346c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50232a98be954d9f9a2e1564566733c5, user_id=7fa5e3cc28ca46d2a4f3b44f23c5a76b, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:21.310636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8813ec-bcea-43e9-91fb-848ddc1346c2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:21.313514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:21.320489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf57ce-31a6-425e-ae43-3894c1e2d7a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:21.324001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf57ce-31a6-425e-ae43-3894c1e2d7a7 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:21.347423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 867/1724] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:21.349160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf57ce-31a6-425e-ae43-3894c1e2d7a7 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:21.350286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 858/1725] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:21 2022] PUT /identity/v3/projects/50232a98be954d9f9a2e1564566733c5/users/7fa5e3cc28ca46d2a4f3b44f23c5a76b/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:21.355690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8813ec-bcea-43e9-91fb-848ddc1346c2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:21.357526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6c8813ec-bcea-43e9-91fb-848ddc1346c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:21.376862 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf57ce-31a6-425e-ae43-3894c1e2d7a7 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:21.558842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:21.562193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923556d0-3a6c-49a5-a246-f7b52d61e2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:21.563625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923556d0-3a6c-49a5-a246-f7b52d61e2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:21.563914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923556d0-3a6c-49a5-a246-f7b52d61e2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:21.564194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923556d0-3a6c-49a5-a246-f7b52d61e2f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:21.568687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923556d0-3a6c-49a5-a246-f7b52d61e2f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:21.569042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8813ec-bcea-43e9-91fb-848ddc1346c2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:21.570957 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923556d0-3a6c-49a5-a246-f7b52d61e2f6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:21.581485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48c188-9f48-4a49-b6a7-ca0a56dcb6d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:21.582123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48c188-9f48-4a49-b6a7-ca0a56dcb6d2 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:21.582406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48c188-9f48-4a49-b6a7-ca0a56dcb6d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:21.585976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48c188-9f48-4a49-b6a7-ca0a56dcb6d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:21.647115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 868/1726] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:21 2022] POST /identity/v3/projects => generated 377 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:21.648112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923556d0-3a6c-49a5-a246-f7b52d61e2f6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:21.652881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-923556d0-3a6c-49a5-a246-f7b52d61e2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:21.709546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48c188-9f48-4a49-b6a7-ca0a56dcb6d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:21.714786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48c188-9f48-4a49-b6a7-ca0a56dcb6d2 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:21.727205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 859/1727] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:21.743904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48c188-9f48-4a49-b6a7-ca0a56dcb6d2 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:21.813084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48c188-9f48-4a49-b6a7-ca0a56dcb6d2 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:21.817228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f93302-39ca-4ebe-ac3c-5c41212847de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:21.817873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f93302-39ca-4ebe-ac3c-5c41212847de service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:21.818155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f93302-39ca-4ebe-ac3c-5c41212847de service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:21.818532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f93302-39ca-4ebe-ac3c-5c41212847de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:21.820433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-76f93302-39ca-4ebe-ac3c-5c41212847de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:21.842192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923556d0-3a6c-49a5-a246-f7b52d61e2f6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:21.855634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c5c10-6781-436f-a8b7-95ce8dd69f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:21.857458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c5c10-6781-436f-a8b7-95ce8dd69f9f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:21.857891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c5c10-6781-436f-a8b7-95ce8dd69f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:21.858179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c5c10-6781-436f-a8b7-95ce8dd69f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:21.932732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f93302-39ca-4ebe-ac3c-5c41212847de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:21.934672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f93302-39ca-4ebe-ac3c-5c41212847de service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:21.960725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 860/1728] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:21.963927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3ea35-20dc-41c0-bd4a-8ee7bf04884e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:21.963927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3ea35-20dc-41c0-bd4a-8ee7bf04884e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:21.964194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3ea35-20dc-41c0-bd4a-8ee7bf04884e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:22.025593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-30a3ea35-20dc-41c0-bd4a-8ee7bf04884e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:22.028491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-30a3ea35-20dc-41c0-bd4a-8ee7bf04884e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:22.038078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c5c10-6781-436f-a8b7-95ce8dd69f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:22.041625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c5c10-6781-436f-a8b7-95ce8dd69f9f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:22.081287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 869/1729] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:22.085335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c5c10-6781-436f-a8b7-95ce8dd69f9f service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:22.168743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:22.197955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 861/1730] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:22.203207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ae5ee-a1fe-46a9-9ab5-c095f16bdbda None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:22.203536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ae5ee-a1fe-46a9-9ab5-c095f16bdbda None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:22.203770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ae5ee-a1fe-46a9-9ab5-c095f16bdbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:22.221257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:22.279351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-585ae5ee-a1fe-46a9-9ab5-c095f16bdbda None None] MFA Rules not processed for user `25db6f7402ae4a14a04fab0a9915ab77`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:22.289675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-585ae5ee-a1fe-46a9-9ab5-c095f16bdbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:22.428070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c5c10-6781-436f-a8b7-95ce8dd69f9f service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:22.460398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 862/1731] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:22.463275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71269d-a51f-44c8-bca4-5cd2c3d6ba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:22.463926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71269d-a51f-44c8-bca4-5cd2c3d6ba1f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:22.464207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71269d-a51f-44c8-bca4-5cd2c3d6ba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:22.464478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71269d-a51f-44c8-bca4-5cd2c3d6ba1f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:22.465151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71269d-a51f-44c8-bca4-5cd2c3d6ba1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:22.473204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71269d-a51f-44c8-bca4-5cd2c3d6ba1f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:22.473296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1455f-4eb6-4a49-9441-e437f1149aed None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:22.473296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1455f-4eb6-4a49-9441-e437f1149aed None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:22.473296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1455f-4eb6-4a49-9441-e437f1149aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:22.553774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 870/1732] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:22 2022] POST /identity/v3/users => generated 295 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:22.557913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71269d-a51f-44c8-bca4-5cd2c3d6ba1f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:22.581224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-59a1455f-4eb6-4a49-9441-e437f1149aed None None] MFA Rules not processed for user `691a9c5e4d534dcc9c26f8cf88cd7fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:22.583028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-59a1455f-4eb6-4a49-9441-e437f1149aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:22.596179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71269d-a51f-44c8-bca4-5cd2c3d6ba1f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:22.611803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3e97e-96ce-462f-a208-13f5f4b84817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:22.612459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e97e-96ce-462f-a208-13f5f4b84817 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:22.612742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e97e-96ce-462f-a208-13f5f4b84817 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:22.613019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3e97e-96ce-462f-a208-13f5f4b84817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:22.688332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 863/1733] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:22.689419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1455f-4eb6-4a49-9441-e437f1149aed None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:22.691182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-59a1455f-4eb6-4a49-9441-e437f1149aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:22.798476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3e97e-96ce-462f-a208-13f5f4b84817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:22.800080 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3e97e-96ce-462f-a208-13f5f4b84817 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:22.826986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 871/1734] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:22.830105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da1aa7-f681-4288-b392-50f14e9baa6b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:22.830609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da1aa7-f681-4288-b392-50f14e9baa6b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:22.831065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da1aa7-f681-4288-b392-50f14e9baa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:22.874024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1455f-4eb6-4a49-9441-e437f1149aed None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:22.882993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ec838-76e6-4751-af80-bbb7614e4da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:22.884639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ec838-76e6-4751-af80-bbb7614e4da6 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:22.884639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ec838-76e6-4751-af80-bbb7614e4da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:22.884834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ec838-76e6-4751-af80-bbb7614e4da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:22.900074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-83da1aa7-f681-4288-b392-50f14e9baa6b None None] MFA Rules not processed for user `25db6f7402ae4a14a04fab0a9915ab77`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:22.901331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83da1aa7-f681-4288-b392-50f14e9baa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:22.910551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ec838-76e6-4751-af80-bbb7614e4da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:22.913707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ec838-76e6-4751-af80-bbb7614e4da6 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:22.927829 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 872/1735] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:22.928773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da1aa7-f681-4288-b392-50f14e9baa6b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:22.951946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 864/1736] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:22.955852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1c6e2-7dc7-4971-9b90-32c2384cf568 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:22.956141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1c6e2-7dc7-4971-9b90-32c2384cf568 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:22.957346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1c6e2-7dc7-4971-9b90-32c2384cf568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:22.958588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da1aa7-f681-4288-b392-50f14e9baa6b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:22.982552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263ab1d-d98e-492d-a76e-33bc5d30f518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:22.983484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263ab1d-d98e-492d-a76e-33bc5d30f518 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:22.983907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263ab1d-d98e-492d-a76e-33bc5d30f518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:22.984322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263ab1d-d98e-492d-a76e-33bc5d30f518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:22.985173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263ab1d-d98e-492d-a76e-33bc5d30f518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:22.987182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263ab1d-d98e-492d-a76e-33bc5d30f518 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:22.990258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-18f1c6e2-7dc7-4971-9b90-32c2384cf568 None None] MFA Rules not processed for user `691a9c5e4d534dcc9c26f8cf88cd7fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:22.991655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-18f1c6e2-7dc7-4971-9b90-32c2384cf568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:23.001117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 865/1737] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:23.002147 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1c6e2-7dc7-4971-9b90-32c2384cf568 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:23.037080 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:23.037485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 873/1738] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:23.041408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263ab1d-d98e-492d-a76e-33bc5d30f518 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:23.121561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1c6e2-7dc7-4971-9b90-32c2384cf568 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:23.130228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b1222-f914-40f6-a6d7-8ea43ff0a5ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:23.131022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b1222-f914-40f6-a6d7-8ea43ff0a5ef service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:23.131309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b1222-f914-40f6-a6d7-8ea43ff0a5ef service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:23.131583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b1222-f914-40f6-a6d7-8ea43ff0a5ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:23.158478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b1222-f914-40f6-a6d7-8ea43ff0a5ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:23.163374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b1222-f914-40f6-a6d7-8ea43ff0a5ef service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:23.181252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 866/1739] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:23.183961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b1222-f914-40f6-a6d7-8ea43ff0a5ef service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:23.196946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b1222-f914-40f6-a6d7-8ea43ff0a5ef service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:23.202697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e32984-0612-44cf-af17-c592f059b82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:23.203552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e32984-0612-44cf-af17-c592f059b82c admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:23.203840 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e32984-0612-44cf-af17-c592f059b82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:23.204311 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e32984-0612-44cf-af17-c592f059b82c admin admin] PATH_INFO: `/v3/projects/def3ff57a65d4cc49640e727f34498f2/users/7a5cb8c2a37042658c78342b612968fc/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:23.223761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e32984-0612-44cf-af17-c592f059b82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=def3ff57a65d4cc49640e727f34498f2, user_id=7a5cb8c2a37042658c78342b612968fc, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:23.226810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e32984-0612-44cf-af17-c592f059b82c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:23.276851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 867/1740] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:23 2022] PUT /identity/v3/projects/def3ff57a65d4cc49640e727f34498f2/users/7a5cb8c2a37042658c78342b612968fc/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:23.277844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e32984-0612-44cf-af17-c592f059b82c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:23.319293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263ab1d-d98e-492d-a76e-33bc5d30f518 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:23.372140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e32984-0612-44cf-af17-c592f059b82c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:23.388786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65690744-1d9b-4a7e-a58a-3ff1e5854565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:23.389642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65690744-1d9b-4a7e-a58a-3ff1e5854565 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:23.390192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65690744-1d9b-4a7e-a58a-3ff1e5854565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:23.390493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65690744-1d9b-4a7e-a58a-3ff1e5854565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:23.391793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65690744-1d9b-4a7e-a58a-3ff1e5854565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:23.395892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65690744-1d9b-4a7e-a58a-3ff1e5854565 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:23.423893 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:23.424824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 868/1741] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:23.428976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65690744-1d9b-4a7e-a58a-3ff1e5854565 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:23.454145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65690744-1d9b-4a7e-a58a-3ff1e5854565 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:23.465763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c00f2-9f23-4c4e-9be9-95d0b4660b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:23.467565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c00f2-9f23-4c4e-9be9-95d0b4660b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:23.468121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c00f2-9f23-4c4e-9be9-95d0b4660b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:23.469341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c00f2-9f23-4c4e-9be9-95d0b4660b7a admin admin] PATH_INFO: `/v3/projects/def3ff57a65d4cc49640e727f34498f2/users/7a5cb8c2a37042658c78342b612968fc/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:23.475879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c00f2-9f23-4c4e-9be9-95d0b4660b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=def3ff57a65d4cc49640e727f34498f2, user_id=7a5cb8c2a37042658c78342b612968fc, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:23.480604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c00f2-9f23-4c4e-9be9-95d0b4660b7a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:23.490038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb300a0a-adda-4da2-939d-9bcc8a2ba836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:23.491438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb300a0a-adda-4da2-939d-9bcc8a2ba836 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:23.492546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb300a0a-adda-4da2-939d-9bcc8a2ba836 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:23.493768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb300a0a-adda-4da2-939d-9bcc8a2ba836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:23.515829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 869/1742] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:23 2022] PUT /identity/v3/projects/def3ff57a65d4cc49640e727f34498f2/users/7a5cb8c2a37042658c78342b612968fc/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:23.527236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6890f3-f435-4d1b-943a-cd74c4cd7792 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:23.527732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6890f3-f435-4d1b-943a-cd74c4cd7792 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:23.528278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6890f3-f435-4d1b-943a-cd74c4cd7792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:23.625366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4b6890f3-f435-4d1b-943a-cd74c4cd7792 None None] MFA Rules not processed for user `7a5cb8c2a37042658c78342b612968fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:23.627367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4b6890f3-f435-4d1b-943a-cd74c4cd7792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:23.659478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb300a0a-adda-4da2-939d-9bcc8a2ba836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:23.668097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb300a0a-adda-4da2-939d-9bcc8a2ba836 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:23.707529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 874/1743] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:23.770542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:23.833820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 870/1744] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:58:23.835534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb300a0a-adda-4da2-939d-9bcc8a2ba836 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:24.058750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb300a0a-adda-4da2-939d-9bcc8a2ba836 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:24.181991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9ff4f-9856-4290-8ad0-8f97223605c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:24.182421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9ff4f-9856-4290-8ad0-8f97223605c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:24.182729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9ff4f-9856-4290-8ad0-8f97223605c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:24.183021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9ff4f-9856-4290-8ad0-8f97223605c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:24.299361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:24.329645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9ff4f-9856-4290-8ad0-8f97223605c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:24.331266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9ff4f-9856-4290-8ad0-8f97223605c4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:24.360062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 875/1745] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:24.401210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a229-3206-45a4-b598-6ff221b93932 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:24.401651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a229-3206-45a4-b598-6ff221b93932 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:24.401827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a229-3206-45a4-b598-6ff221b93932 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:24.403475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 871/1746] 10.222.0.20 () {54 vars in 912 bytes} [Sat Jul 30 04:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:24.409662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fbebe-6509-4585-9f2a-cd2d619c109f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:24.409952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fbebe-6509-4585-9f2a-cd2d619c109f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:24.410235 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fbebe-6509-4585-9f2a-cd2d619c109f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:24.506866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cd5fbebe-6509-4585-9f2a-cd2d619c109f None None] MFA Rules not processed for user `426be56424504e958402d785a068d2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:24.508092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd5fbebe-6509-4585-9f2a-cd2d619c109f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:24.646664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 876/1747] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 04:58:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:58:24.795637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5a229-3206-45a4-b598-6ff221b93932 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:24.797454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fac5a229-3206-45a4-b598-6ff221b93932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:24.828736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5a229-3206-45a4-b598-6ff221b93932 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:24.836873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89037a21-f9f2-46c0-9dbf-73dc5a610026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:24.837761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89037a21-f9f2-46c0-9dbf-73dc5a610026 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:24.838188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89037a21-f9f2-46c0-9dbf-73dc5a610026 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:24.838851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89037a21-f9f2-46c0-9dbf-73dc5a610026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:24.852526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89037a21-f9f2-46c0-9dbf-73dc5a610026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:24.860419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89037a21-f9f2-46c0-9dbf-73dc5a610026 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:24.883961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 872/1748] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:25.197235 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a7b56-7cc5-42ed-9a08-e738ba3110ac None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:25.197696 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a7b56-7cc5-42ed-9a08-e738ba3110ac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:25.197768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a7b56-7cc5-42ed-9a08-e738ba3110ac None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:25.199597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 877/1749] 10.222.0.20 () {54 vars in 912 bytes} [Sat Jul 30 04:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:25.212638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd8a7d-c1d3-47bf-9fa6-a5a4344bc41c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:25.212998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd8a7d-c1d3-47bf-9fa6-a5a4344bc41c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:25.213437 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd8a7d-c1d3-47bf-9fa6-a5a4344bc41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:25.251561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1ffd8a7d-c1d3-47bf-9fa6-a5a4344bc41c None None] MFA Rules not processed for user `426be56424504e958402d785a068d2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:25.252793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1ffd8a7d-c1d3-47bf-9fa6-a5a4344bc41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:25.276360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 873/1750] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 04:58:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:25.703631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a7b56-7cc5-42ed-9a08-e738ba3110ac None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:25.724296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a7b56-7cc5-42ed-9a08-e738ba3110ac None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:25.743897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee04ee0-a0f0-4eff-a714-e999ad8f0c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:25.743897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04ee0-a0f0-4eff-a714-e999ad8f0c78 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:25.743897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04ee0-a0f0-4eff-a714-e999ad8f0c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:25.743897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04ee0-a0f0-4eff-a714-e999ad8f0c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:25.756566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee04ee0-a0f0-4eff-a714-e999ad8f0c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:25.760541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee04ee0-a0f0-4eff-a714-e999ad8f0c78 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:25.778350 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 878/1751] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:58:26.246902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e10589-9151-4c60-82a8-c146b480e814 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:26.247805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e10589-9151-4c60-82a8-c146b480e814 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:26.248227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e10589-9151-4c60-82a8-c146b480e814 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:26.249807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 874/1752] 10.222.0.20 () {54 vars in 961 bytes} [Sat Jul 30 04:58:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 30 04:58:26.256304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d689a-dcd7-41a2-83eb-4cba35238df6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:26.257178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d689a-dcd7-41a2-83eb-4cba35238df6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:26.257726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d689a-dcd7-41a2-83eb-4cba35238df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:26.292872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-228d689a-dcd7-41a2-83eb-4cba35238df6 None None] MFA Rules not processed for user `21b7c64636bd47e6b9a486b6cdaab4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:26.294402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-228d689a-dcd7-41a2-83eb-4cba35238df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:26.421966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 879/1753] 10.222.0.20 () {58 vars in 1058 bytes} [Sat Jul 30 04:58:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:26.429510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e10589-9151-4c60-82a8-c146b480e814 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:26.459049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e10589-9151-4c60-82a8-c146b480e814 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:26.472661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72160f9f-9b6b-4890-b6b4-b3f4e6bfc182 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:26.473575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72160f9f-9b6b-4890-b6b4-b3f4e6bfc182 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:26.474527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72160f9f-9b6b-4890-b6b4-b3f4e6bfc182 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:26.476133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72160f9f-9b6b-4890-b6b4-b3f4e6bfc182 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:26.481535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-72160f9f-9b6b-4890-b6b4-b3f4e6bfc182 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:26.507349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72160f9f-9b6b-4890-b6b4-b3f4e6bfc182 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:26.511205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72160f9f-9b6b-4890-b6b4-b3f4e6bfc182 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:26.543484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 875/1754] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:26.579706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1a2af-399b-4013-8d8c-1b84b65508ed None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:26.580007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1a2af-399b-4013-8d8c-1b84b65508ed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:26.580281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1a2af-399b-4013-8d8c-1b84b65508ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:26.722599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b7e1a2af-399b-4013-8d8c-1b84b65508ed None None] MFA Rules not processed for user `7fa5e3cc28ca46d2a4f3b44f23c5a76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:26.724937 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b7e1a2af-399b-4013-8d8c-1b84b65508ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:26.823797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f17db4-ea89-4b26-a38a-471bed972d24 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:26.824090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f17db4-ea89-4b26-a38a-471bed972d24 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:26.824396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f17db4-ea89-4b26-a38a-471bed972d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:26.923874 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-45f17db4-ea89-4b26-a38a-471bed972d24 None None] MFA Rules not processed for user `25db6f7402ae4a14a04fab0a9915ab77`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:26.925027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-45f17db4-ea89-4b26-a38a-471bed972d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:26.983052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:27.047241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 880/1755] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:27.048453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1a2af-399b-4013-8d8c-1b84b65508ed None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:27.163255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 876/1756] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:27.308311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1a2af-399b-4013-8d8c-1b84b65508ed None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:27.328174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd9e27-f5d9-4ada-bb21-f740ae581623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:27.328830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd9e27-f5d9-4ada-bb21-f740ae581623 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:27.329135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd9e27-f5d9-4ada-bb21-f740ae581623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:27.329422 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd9e27-f5d9-4ada-bb21-f740ae581623 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:27.338983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd9e27-f5d9-4ada-bb21-f740ae581623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:27.350945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd9e27-f5d9-4ada-bb21-f740ae581623 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:27.470339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 881/1757] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:27 2022] POST /identity/v3/projects => generated 403 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:27.474272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f17db4-ea89-4b26-a38a-471bed972d24 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:27.492722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f17db4-ea89-4b26-a38a-471bed972d24 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:27.655175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239a40c-48fb-4d27-b201-fd27afa4ffe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:27.656040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a40c-48fb-4d27-b201-fd27afa4ffe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:27.656416 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a40c-48fb-4d27-b201-fd27afa4ffe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:27.656777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239a40c-48fb-4d27-b201-fd27afa4ffe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:27.657676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239a40c-48fb-4d27-b201-fd27afa4ffe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:27.660197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239a40c-48fb-4d27-b201-fd27afa4ffe6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:27.733966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd9e27-f5d9-4ada-bb21-f740ae581623 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:27.736003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 877/1758] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:27 2022] POST /identity/v3/users => generated 308 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:58:27.746705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd9e27-f5d9-4ada-bb21-f740ae581623 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:27.760995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f863f0-8091-49a4-bb2c-a7a92889aed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:27.762004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f863f0-8091-49a4-bb2c-a7a92889aed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:27.762425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f863f0-8091-49a4-bb2c-a7a92889aed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:27.762857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f863f0-8091-49a4-bb2c-a7a92889aed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:27.764187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f863f0-8091-49a4-bb2c-a7a92889aed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:27.784163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f863f0-8091-49a4-bb2c-a7a92889aed6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:27.830396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239a40c-48fb-4d27-b201-fd27afa4ffe6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:27.860308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239a40c-48fb-4d27-b201-fd27afa4ffe6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:27.864735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 882/1759] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:27.882636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad1c01-c267-49b3-825b-5cb34e184fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:27.884456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad1c01-c267-49b3-825b-5cb34e184fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:27.885712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad1c01-c267-49b3-825b-5cb34e184fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:27.886176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad1c01-c267-49b3-825b-5cb34e184fa4 admin admin] PATH_INFO: `/v3/projects/5b45753b4fff4359b29363358307789c/users/afb46970b3d54c8e8f8a4e76f49de422/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:27.909443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f863f0-8091-49a4-bb2c-a7a92889aed6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:27.940833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f863f0-8091-49a4-bb2c-a7a92889aed6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:27.982454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:27.987656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:27.988089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:27.989661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] PATH_INFO: `/v3/users/0a24c464748a4d2882eba3536855f9e3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:27.991433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a24c464748a4d2882eba3536855f9e3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:28.003508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:28.006825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad1c01-c267-49b3-825b-5cb34e184fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b45753b4fff4359b29363358307789c, user_id=afb46970b3d54c8e8f8a4e76f49de422, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:28.016098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad1c01-c267-49b3-825b-5cb34e184fa4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:28.054499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 878/1760] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:27 2022] PUT /identity/v3/projects/5b45753b4fff4359b29363358307789c/users/afb46970b3d54c8e8f8a4e76f49de422/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:28.056291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad1c01-c267-49b3-825b-5cb34e184fa4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:28.084010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:28.207946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a24c464748a4d2882eba3536855f9e3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:28.221218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad1c01-c267-49b3-825b-5cb34e184fa4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:28.243752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0033b29-d1e5-4c9a-9b16-d62f3e94d5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:28.247589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0033b29-d1e5-4c9a-9b16-d62f3e94d5fc admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:28.247884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0033b29-d1e5-4c9a-9b16-d62f3e94d5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:28.248247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0033b29-d1e5-4c9a-9b16-d62f3e94d5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:28.255444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0033b29-d1e5-4c9a-9b16-d62f3e94d5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:28.258111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0033b29-d1e5-4c9a-9b16-d62f3e94d5fc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:28.262987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a24c464748a4d2882eba3536855f9e3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:28.266958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a24c464748a4d2882eba3536855f9e3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:28.268304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a24c464748a4d2882eba3536855f9e3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:28.278120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 879/1761] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:28.280973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0033b29-d1e5-4c9a-9b16-d62f3e94d5fc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:28.283904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f0033b29-d1e5-4c9a-9b16-d62f3e94d5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:28.325573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0033b29-d1e5-4c9a-9b16-d62f3e94d5fc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:28.347260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be823d-841e-4a43-b796-ba521e6bae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:28.348006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be823d-841e-4a43-b796-ba521e6bae6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:28.348289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be823d-841e-4a43-b796-ba521e6bae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:28.348564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be823d-841e-4a43-b796-ba521e6bae6e admin admin] PATH_INFO: `/v3/projects/5b45753b4fff4359b29363358307789c/users/afb46970b3d54c8e8f8a4e76f49de422/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:28.354690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be823d-841e-4a43-b796-ba521e6bae6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b45753b4fff4359b29363358307789c, user_id=afb46970b3d54c8e8f8a4e76f49de422, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:28.363108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e633ddd1-f850-48c8-8c39-457bf6f0d9cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a24c464748a4d2882eba3536855f9e3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:28.365773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be823d-841e-4a43-b796-ba521e6bae6e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:28.409783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 880/1762] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:28 2022] PUT /identity/v3/projects/5b45753b4fff4359b29363358307789c/users/afb46970b3d54c8e8f8a4e76f49de422/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:28.413474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7226f2-25b6-4eeb-9d15-778eecbd7f36 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:28.413775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7226f2-25b6-4eeb-9d15-778eecbd7f36 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:28.414087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7226f2-25b6-4eeb-9d15-778eecbd7f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:28.465954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 883/1763] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:27 2022] DELETE /identity/v3/users/0a24c464748a4d2882eba3536855f9e3 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:58:28.467125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa6f62-e94b-4d99-9843-2bf2acd19662 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:28.467423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa6f62-e94b-4d99-9843-2bf2acd19662 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:28.467705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa6f62-e94b-4d99-9843-2bf2acd19662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:28.499576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2d7226f2-25b6-4eeb-9d15-778eecbd7f36 None None] MFA Rules not processed for user `a8abb58f9c274e568d2d0fe3d4df11de`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:28.500854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2d7226f2-25b6-4eeb-9d15-778eecbd7f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:28.572507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-39aa6f62-e94b-4d99-9843-2bf2acd19662 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:28.579389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39aa6f62-e94b-4d99-9843-2bf2acd19662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:28.661943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 881/1764] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:28.748052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:28.853135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 884/1765] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:28.897903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7226f2-25b6-4eeb-9d15-778eecbd7f36 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:28.899917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2d7226f2-25b6-4eeb-9d15-778eecbd7f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:29.144322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7226f2-25b6-4eeb-9d15-778eecbd7f36 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:29.153942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8d524-8507-4da0-9dc0-b237b5cea186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:29.154723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8d524-8507-4da0-9dc0-b237b5cea186 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:29.155012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8d524-8507-4da0-9dc0-b237b5cea186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:29.155296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8d524-8507-4da0-9dc0-b237b5cea186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:29.187662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8d524-8507-4da0-9dc0-b237b5cea186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:29.189160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8d524-8507-4da0-9dc0-b237b5cea186 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:29.227229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 882/1766] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:29.790423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137262b-c682-4800-8d96-da4aeba663d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:29.792136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137262b-c682-4800-8d96-da4aeba663d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:29.793068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137262b-c682-4800-8d96-da4aeba663d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:29.838485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3137262b-c682-4800-8d96-da4aeba663d1 None None] MFA Rules not processed for user `d3a56c6d44b74429a6616cdfc2869b71`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:29.843349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3137262b-c682-4800-8d96-da4aeba663d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:30.035427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 885/1767] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:30.786811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8d524-8507-4da0-9dc0-b237b5cea186 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:30.788437 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-43f8d524-8507-4da0-9dc0-b237b5cea186 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:30.938557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137262b-c682-4800-8d96-da4aeba663d1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:30.940133 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3137262b-c682-4800-8d96-da4aeba663d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:30.978656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8d524-8507-4da0-9dc0-b237b5cea186 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:30.984102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45078b1d-890e-4fa8-8830-074c5eab399b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:30.985571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45078b1d-890e-4fa8-8830-074c5eab399b service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:30.986021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45078b1d-890e-4fa8-8830-074c5eab399b service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:30.986653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45078b1d-890e-4fa8-8830-074c5eab399b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:30.988836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-45078b1d-890e-4fa8-8830-074c5eab399b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:30.996269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:31.001159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137262b-c682-4800-8d96-da4aeba663d1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:31.047115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:31.052556 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:31.052852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:31.053128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] PATH_INFO: `/v3/projects/e96985506fe649b694f9b87979900cd9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:31.060132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96985506fe649b694f9b87979900cd9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:31.062122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:31.169478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96985506fe649b694f9b87979900cd9', 'request_id': 'req-11cd75d1-ff94-4ea6-9f76-add206132217'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.180779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45078b1d-890e-4fa8-8830-074c5eab399b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:31.182214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45078b1d-890e-4fa8-8830-074c5eab399b service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:31.196913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96985506fe649b694f9b87979900cd9', 'request_id': 'req-11cd75d1-ff94-4ea6-9f76-add206132217'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.206543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96985506fe649b694f9b87979900cd9', 'request_id': 'req-11cd75d1-ff94-4ea6-9f76-add206132217'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.219301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 883/1768] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:31.222868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642e565-9ca7-44c3-a6db-c8e646461142 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:31.223154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642e565-9ca7-44c3-a6db-c8e646461142 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:31.223433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642e565-9ca7-44c3-a6db-c8e646461142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:31.247179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] The token cache is being invalidate because project e96985506fe649b694f9b87979900cd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:31.247473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.249408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.252403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 886/1769] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:30 2022] DELETE /identity/v3/projects/e96985506fe649b694f9b87979900cd9 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:31.258420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:31.261212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:31.307830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a642e565-9ca7-44c3-a6db-c8e646461142 None None] MFA Rules not processed for user `afb46970b3d54c8e8f8a4e76f49de422`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:31.308202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a642e565-9ca7-44c3-a6db-c8e646461142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:31.420013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd75d1-ff94-4ea6-9f76-add206132217 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:31.430361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:31.431605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:31.431884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:31.432156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] PATH_INFO: `/v3/users/36aa2484e60f4de0934f2aa1667e12df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:31.435797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36aa2484e60f4de0934f2aa1667e12df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:31.440608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:31.459280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:31.475909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 884/1770] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:31 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:31.535138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36aa2484e60f4de0934f2aa1667e12df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.549597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36aa2484e60f4de0934f2aa1667e12df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.556723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36aa2484e60f4de0934f2aa1667e12df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.557484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36aa2484e60f4de0934f2aa1667e12df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.582773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c4d944b-4955-4c85-bc8c-65f498715885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36aa2484e60f4de0934f2aa1667e12df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:31.608454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 887/1771] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:31 2022] DELETE /identity/v3/users/36aa2484e60f4de0934f2aa1667e12df => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:32.385128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642e565-9ca7-44c3-a6db-c8e646461142 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:32.396838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642e565-9ca7-44c3-a6db-c8e646461142 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:32.504509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:32.505534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:32.506144 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:32.506596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] PATH_INFO: `/v3/projects/361c2261b9134891b6aac5cd2762ea01` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:32.508556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=361c2261b9134891b6aac5cd2762ea01)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:32.510808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:32.549370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:32.584439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '361c2261b9134891b6aac5cd2762ea01', 'request_id': 'req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:32.603654 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '361c2261b9134891b6aac5cd2762ea01', 'request_id': 'req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:32.622471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '361c2261b9134891b6aac5cd2762ea01', 'request_id': 'req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:32.638965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] The token cache is being invalidate because project 361c2261b9134891b6aac5cd2762ea01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:32.639378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:32.641198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5db29d22-aa05-4ac2-ab35-1d36db3d33d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:32.648477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 885/1772] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:32 2022] DELETE /identity/v3/projects/361c2261b9134891b6aac5cd2762ea01 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:32.649966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8ff6-25fd-447f-9e88-214c4a40aa09 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:32.650427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8ff6-25fd-447f-9e88-214c4a40aa09 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:32.650559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8ff6-25fd-447f-9e88-214c4a40aa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:32.688921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-229c8ff6-25fd-447f-9e88-214c4a40aa09 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:32.690167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-229c8ff6-25fd-447f-9e88-214c4a40aa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:32.715747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4b494-3618-4147-91cd-5e34556698d8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:32.716098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4b494-3618-4147-91cd-5e34556698d8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:32.716971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4b494-3618-4147-91cd-5e34556698d8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:32.718686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 886/1773] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 04:58:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:32.750327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1a68d-8ec9-4a70-afae-685b63c3eeba None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:32.751026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1a68d-8ec9-4a70-afae-685b63c3eeba None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:32.751429 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1a68d-8ec9-4a70-afae-685b63c3eeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:32.791152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-36a1a68d-8ec9-4a70-afae-685b63c3eeba None None] MFA Rules not processed for user `21b7c64636bd47e6b9a486b6cdaab4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:32.792475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-36a1a68d-8ec9-4a70-afae-685b63c3eeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:32.799381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 888/1774] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:32.808575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42535ac1-55f0-41de-9a10-e485438e1586 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:32.808873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42535ac1-55f0-41de-9a10-e485438e1586 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:32.809154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42535ac1-55f0-41de-9a10-e485438e1586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:32.867976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-42535ac1-55f0-41de-9a10-e485438e1586 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:32.869473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42535ac1-55f0-41de-9a10-e485438e1586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:32.888342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 889/1775] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:32.889497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42535ac1-55f0-41de-9a10-e485438e1586 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:32.905297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42535ac1-55f0-41de-9a10-e485438e1586 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:32.913767 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:32.915466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:32.915973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:32.916128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:32.916832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:32.917525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:32.919320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:32.933158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 887/1776] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 04:58:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:32.941177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 890/1777] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:32.948009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1a68d-8ec9-4a70-afae-685b63c3eeba None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:32.971465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1a68d-8ec9-4a70-afae-685b63c3eeba None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:32.983012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2598305-34f1-4acb-b7af-77c5438ec12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:32.983664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598305-34f1-4acb-b7af-77c5438ec12b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:32.984022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598305-34f1-4acb-b7af-77c5438ec12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:32.984202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598305-34f1-4acb-b7af-77c5438ec12b admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:32.984882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2598305-34f1-4acb-b7af-77c5438ec12b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:32.989080 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2598305-34f1-4acb-b7af-77c5438ec12b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:33.013372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:33.015221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:33.040772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c4ca2-de86-421e-aed2-6c58bd95eba0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:33.057520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb046a2d-09f2-4ab0-b858-adc7d5846db0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:33.058055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 888/1778] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:32 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:33.059217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb046a2d-09f2-4ab0-b858-adc7d5846db0 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:33.059601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb046a2d-09f2-4ab0-b858-adc7d5846db0 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:33.059978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb046a2d-09f2-4ab0-b858-adc7d5846db0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:33.061581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2598305-34f1-4acb-b7af-77c5438ec12b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:33.084626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb046a2d-09f2-4ab0-b858-adc7d5846db0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:33.086883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb046a2d-09f2-4ab0-b858-adc7d5846db0 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:33.115357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 891/1779] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:33.186857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2598305-34f1-4acb-b7af-77c5438ec12b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:33.198575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a60d37-9f30-407f-acd4-e9142043a217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:33.199453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a60d37-9f30-407f-acd4-e9142043a217 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:33.199863 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a60d37-9f30-407f-acd4-e9142043a217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:33.200134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a60d37-9f30-407f-acd4-e9142043a217 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:33.200831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a60d37-9f30-407f-acd4-e9142043a217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:33.205565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a60d37-9f30-407f-acd4-e9142043a217 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:33.245464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 889/1780] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:33 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:33.246843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb046a2d-09f2-4ab0-b858-adc7d5846db0 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:33.261389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb046a2d-09f2-4ab0-b858-adc7d5846db0 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:33.266531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eaeca-492e-412e-befa-fde486bc8d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:33.267404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eaeca-492e-412e-befa-fde486bc8d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:33.267890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eaeca-492e-412e-befa-fde486bc8d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:33.268388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eaeca-492e-412e-befa-fde486bc8d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:33.269406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758eaeca-492e-412e-befa-fde486bc8d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:33.275065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758eaeca-492e-412e-befa-fde486bc8d9d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:33.292227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 892/1781] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:33.296497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a60d37-9f30-407f-acd4-e9142043a217 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:33.307181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a60d37-9f30-407f-acd4-e9142043a217 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:33.311929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5ea8b-cc6c-4842-8283-7a8d8420c8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:33.312543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5ea8b-cc6c-4842-8283-7a8d8420c8ad admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:33.312826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5ea8b-cc6c-4842-8283-7a8d8420c8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:33.313104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5ea8b-cc6c-4842-8283-7a8d8420c8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:33.313804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5ea8b-cc6c-4842-8283-7a8d8420c8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:33.315433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5ea8b-cc6c-4842-8283-7a8d8420c8ad admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:33.330547 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 890/1782] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:33.332815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eaeca-492e-412e-befa-fde486bc8d9d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:33.352961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eaeca-492e-412e-befa-fde486bc8d9d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:33.395627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda073d-db9f-4d5b-84f7-430c5319193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:33.396318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda073d-db9f-4d5b-84f7-430c5319193d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:33.396599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda073d-db9f-4d5b-84f7-430c5319193d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:33.396877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda073d-db9f-4d5b-84f7-430c5319193d admin admin] PATH_INFO: `/v3/projects/2ffc60ac499b456f8ec93a9d57c14df3/users/ccde0065abf8481b969790329cc4ca77/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:33.450418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda073d-db9f-4d5b-84f7-430c5319193d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffc60ac499b456f8ec93a9d57c14df3, user_id=ccde0065abf8481b969790329cc4ca77, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:33.453553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda073d-db9f-4d5b-84f7-430c5319193d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:33.484754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 893/1783] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:33 2022] PUT /identity/v3/projects/2ffc60ac499b456f8ec93a9d57c14df3/users/ccde0065abf8481b969790329cc4ca77/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:58:33.496367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910533b0-2572-4113-bbe0-12a242feb5bb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:33.496367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910533b0-2572-4113-bbe0-12a242feb5bb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:33.497021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910533b0-2572-4113-bbe0-12a242feb5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:33.540518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:33.562118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-910533b0-2572-4113-bbe0-12a242feb5bb None None] MFA Rules not processed for user `ccde0065abf8481b969790329cc4ca77`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:33.563421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-910533b0-2572-4113-bbe0-12a242feb5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:33.743594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 891/1784] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:33.748812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda073d-db9f-4d5b-84f7-430c5319193d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:33.811208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:33.880521 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda073d-db9f-4d5b-84f7-430c5319193d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:33.891213 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224d8e7-7db3-4b98-956a-6ded9d87e9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:33.893540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d8e7-7db3-4b98-956a-6ded9d87e9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:33.894091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d8e7-7db3-4b98-956a-6ded9d87e9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:33.894606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224d8e7-7db3-4b98-956a-6ded9d87e9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:33.896408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224d8e7-7db3-4b98-956a-6ded9d87e9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:33.899728 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224d8e7-7db3-4b98-956a-6ded9d87e9c1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:33.941742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 894/1785] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:33 2022] POST /identity/v3/projects => generated 393 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:33.946686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910533b0-2572-4113-bbe0-12a242feb5bb None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:33.960105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910533b0-2572-4113-bbe0-12a242feb5bb None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:33.998853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a364245-7489-4501-8605-b2979fa3fbc4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:33.999469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a364245-7489-4501-8605-b2979fa3fbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.001227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a364245-7489-4501-8605-b2979fa3fbc4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.003951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 895/1786] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 04:58:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:34.029283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f532b-ba28-409f-85b5-c58eaca6a284 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.029605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f532b-ba28-409f-85b5-c58eaca6a284 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.029967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f532b-ba28-409f-85b5-c58eaca6a284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.069675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1854a8-c21d-497b-8f6a-71f074b787d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:34.069675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1854a8-c21d-497b-8f6a-71f074b787d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.069675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1854a8-c21d-497b-8f6a-71f074b787d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.069675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1854a8-c21d-497b-8f6a-71f074b787d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.070101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1854a8-c21d-497b-8f6a-71f074b787d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:34.070169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ad7f532b-ba28-409f-85b5-c58eaca6a284 None None] MFA Rules not processed for user `21b7c64636bd47e6b9a486b6cdaab4b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:34.070169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ad7f532b-ba28-409f-85b5-c58eaca6a284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:34.077205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1854a8-c21d-497b-8f6a-71f074b787d5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:34.118037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 892/1787] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:33 2022] POST /identity/v3/users => generated 303 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:34.122076 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1854a8-c21d-497b-8f6a-71f074b787d5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:34.128304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5f1854a8-c21d-497b-8f6a-71f074b787d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:34.149120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1854a8-c21d-497b-8f6a-71f074b787d5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:34.155222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:34.156214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.156697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.157185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] PATH_INFO: `/v3/users/7fa5e3cc28ca46d2a4f3b44f23c5a76b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.159166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa5e3cc28ca46d2a4f3b44f23c5a76b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:34.161473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:34.195218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 896/1788] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 04:58:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:34.197475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f532b-ba28-409f-85b5-c58eaca6a284 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:34.213276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f532b-ba28-409f-85b5-c58eaca6a284 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:34.219192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e0d86-4cad-4122-81eb-ba10bd7b822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:34.219911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e0d86-4cad-4122-81eb-ba10bd7b822b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.220311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e0d86-4cad-4122-81eb-ba10bd7b822b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.220774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e0d86-4cad-4122-81eb-ba10bd7b822b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.221680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e0d86-4cad-4122-81eb-ba10bd7b822b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:34.223536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e0d86-4cad-4122-81eb-ba10bd7b822b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:34.238590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 897/1789] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:34.239729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e0d86-4cad-4122-81eb-ba10bd7b822b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:34.241569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e28e0d86-4cad-4122-81eb-ba10bd7b822b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:34.249911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa5e3cc28ca46d2a4f3b44f23c5a76b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:34.258303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e0d86-4cad-4122-81eb-ba10bd7b822b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:34.264096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:34.265242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa5e3cc28ca46d2a4f3b44f23c5a76b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:34.265406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.266204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.266204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.266887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa5e3cc28ca46d2a4f3b44f23c5a76b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:34.267721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa5e3cc28ca46d2a4f3b44f23c5a76b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:34.268478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:34.287202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:34.290225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:34.291489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa5e3cc28ca46d2a4f3b44f23c5a76b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:34.307628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 898/1790] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:34.312949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:34.323452 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 893/1791] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:34 2022] DELETE /identity/v3/users/7fa5e3cc28ca46d2a4f3b44f23c5a76b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:34.336751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:34.339886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:34.365671 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5077a6-056f-46c3-9860-c3e73faf8a47 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:34.490880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b556f-a346-4b2d-ac1a-9ba6b0e287e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:34.491660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b556f-a346-4b2d-ac1a-9ba6b0e287e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.492014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b556f-a346-4b2d-ac1a-9ba6b0e287e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.493043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b556f-a346-4b2d-ac1a-9ba6b0e287e0 admin admin] PATH_INFO: `/v3/projects/c91f5c0afece492b83f02cfbfd2bb09c/users/2a5e95d693a84ebcaae721f838403c92/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.516080 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b556f-a346-4b2d-ac1a-9ba6b0e287e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91f5c0afece492b83f02cfbfd2bb09c, user_id=2a5e95d693a84ebcaae721f838403c92, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:34.519414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b556f-a346-4b2d-ac1a-9ba6b0e287e0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:34.542782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 899/1792] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:34 2022] PUT /identity/v3/projects/c91f5c0afece492b83f02cfbfd2bb09c/users/2a5e95d693a84ebcaae721f838403c92/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:34.547363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:34.572139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388ec8c-27bf-4aec-996c-ba98837d761a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:34.660652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b556f-a346-4b2d-ac1a-9ba6b0e287e0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:34.662501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-247b556f-a346-4b2d-ac1a-9ba6b0e287e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:34.672069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:34.694949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ad568-4753-4a17-a2e6-4646b3f6b7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:34.696118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ad568-4753-4a17-a2e6-4646b3f6b7be admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.696399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ad568-4753-4a17-a2e6-4646b3f6b7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.696674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ad568-4753-4a17-a2e6-4646b3f6b7be admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.697560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ad568-4753-4a17-a2e6-4646b3f6b7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:34.700853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ad568-4753-4a17-a2e6-4646b3f6b7be admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:34.718342 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 894/1793] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:34.720560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ad568-4753-4a17-a2e6-4646b3f6b7be admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:34.737082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ad568-4753-4a17-a2e6-4646b3f6b7be admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:34.744185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afd946-5102-44f8-a34f-86c75007ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:34.745137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afd946-5102-44f8-a34f-86c75007ad98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.745137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afd946-5102-44f8-a34f-86c75007ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.745548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afd946-5102-44f8-a34f-86c75007ad98 admin admin] PATH_INFO: `/v3/projects/c91f5c0afece492b83f02cfbfd2bb09c/users/2a5e95d693a84ebcaae721f838403c92/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.747669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afd946-5102-44f8-a34f-86c75007ad98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91f5c0afece492b83f02cfbfd2bb09c, user_id=2a5e95d693a84ebcaae721f838403c92, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:34.750831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afd946-5102-44f8-a34f-86c75007ad98 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:34.764052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b556f-a346-4b2d-ac1a-9ba6b0e287e0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:34.765108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 895/1794] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:34 2022] PUT /identity/v3/projects/c91f5c0afece492b83f02cfbfd2bb09c/users/2a5e95d693a84ebcaae721f838403c92/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:34.776176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dae6ab-20fe-41a6-957b-b0ce0cf9ab10 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.776538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dae6ab-20fe-41a6-957b-b0ce0cf9ab10 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.776886 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dae6ab-20fe-41a6-957b-b0ce0cf9ab10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.814307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-16dae6ab-20fe-41a6-957b-b0ce0cf9ab10 None None] MFA Rules not processed for user `2a5e95d693a84ebcaae721f838403c92`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:34.815865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-16dae6ab-20fe-41a6-957b-b0ce0cf9ab10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:34.877437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:34.880022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:34.880334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:34.880619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] PATH_INFO: `/v3/projects/50232a98be954d9f9a2e1564566733c5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:34.882651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50232a98be954d9f9a2e1564566733c5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:34.886051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:34.911006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 896/1795] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:34.934100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dae6ab-20fe-41a6-957b-b0ce0cf9ab10 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:34.935834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-16dae6ab-20fe-41a6-957b-b0ce0cf9ab10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:34.960966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50232a98be954d9f9a2e1564566733c5', 'request_id': 'req-9cb247e3-c064-4077-b138-4bd6b67b3358'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:34.971416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50232a98be954d9f9a2e1564566733c5', 'request_id': 'req-9cb247e3-c064-4077-b138-4bd6b67b3358'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:34.980442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50232a98be954d9f9a2e1564566733c5', 'request_id': 'req-9cb247e3-c064-4077-b138-4bd6b67b3358'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.002269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] The token cache is being invalidate because project 50232a98be954d9f9a2e1564566733c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:35.002603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.003396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.005662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 900/1796] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:34 2022] DELETE /identity/v3/projects/50232a98be954d9f9a2e1564566733c5 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:35.009483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:35.013055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:35.051248 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dae6ab-20fe-41a6-957b-b0ce0cf9ab10 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:35.080023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:35.100002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:35.107527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb247e3-c064-4077-b138-4bd6b67b3358 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:35.114313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:35.115460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:35.115811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:35.116108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] PATH_INFO: `/v3/users/afb46970b3d54c8e8f8a4e76f49de422` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:35.118974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb46970b3d54c8e8f8a4e76f49de422)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:35.122179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:35.158227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0e37-e41d-4b31-83bd-f40d37a9bd36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:35.159150 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0e37-e41d-4b31-83bd-f40d37a9bd36 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:35.159523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0e37-e41d-4b31-83bd-f40d37a9bd36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:35.170766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0e37-e41d-4b31-83bd-f40d37a9bd36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:35.174272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-555c0e37-e41d-4b31-83bd-f40d37a9bd36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:35.213282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb46970b3d54c8e8f8a4e76f49de422'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.235571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb46970b3d54c8e8f8a4e76f49de422'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.236327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb46970b3d54c8e8f8a4e76f49de422'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.236971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb46970b3d54c8e8f8a4e76f49de422'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.271835 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb46970b3d54c8e8f8a4e76f49de422'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.302637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0e37-e41d-4b31-83bd-f40d37a9bd36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:35.304616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0e37-e41d-4b31-83bd-f40d37a9bd36 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:35.305360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 901/1797] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:35 2022] DELETE /identity/v3/users/afb46970b3d54c8e8f8a4e76f49de422 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:58:35.372914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 897/1798] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:35.453272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:35.454736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:35.580562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573955e-1475-4a44-b53d-69304472b013 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:35.589836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cc6dc-37d5-4fc1-8297-fa12bdaeb2c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:35.590958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cc6dc-37d5-4fc1-8297-fa12bdaeb2c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:35.591414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cc6dc-37d5-4fc1-8297-fa12bdaeb2c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:35.591834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cc6dc-37d5-4fc1-8297-fa12bdaeb2c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:35.594620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-046cc6dc-37d5-4fc1-8297-fa12bdaeb2c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:35.656176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0e37-e41d-4b31-83bd-f40d37a9bd36 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:35.667351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0e37-e41d-4b31-83bd-f40d37a9bd36 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:35.687035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046cc6dc-37d5-4fc1-8297-fa12bdaeb2c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:35.688479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046cc6dc-37d5-4fc1-8297-fa12bdaeb2c2 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:35.706580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 902/1799] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:35.741129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:35.741785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:35.742060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:35.742804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] PATH_INFO: `/v3/projects/5b45753b4fff4359b29363358307789c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:35.744201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b45753b4fff4359b29363358307789c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:35.746418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:35.832145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b45753b4fff4359b29363358307789c', 'request_id': 'req-e2442046-248e-44d8-8cf0-4a9989f0ca2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.844485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b45753b4fff4359b29363358307789c', 'request_id': 'req-e2442046-248e-44d8-8cf0-4a9989f0ca2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.852758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b45753b4fff4359b29363358307789c', 'request_id': 'req-e2442046-248e-44d8-8cf0-4a9989f0ca2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.866735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] The token cache is being invalidate because project 5b45753b4fff4359b29363358307789c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:35.867048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.868466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2442046-248e-44d8-8cf0-4a9989f0ca2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:35.870348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 898/1800] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:35 2022] DELETE /identity/v3/projects/5b45753b4fff4359b29363358307789c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:58:35.875440 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b375abe-0290-4293-8a34-6dd9c862c8a9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:35.875981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b375abe-0290-4293-8a34-6dd9c862c8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:35.876333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b375abe-0290-4293-8a34-6dd9c862c8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:35.906393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2b375abe-0290-4293-8a34-6dd9c862c8a9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:35.907680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b375abe-0290-4293-8a34-6dd9c862c8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:35.926293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:35.991704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 903/1801] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:36.001687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe7e86-8347-44ba-9963-77bb3519b66d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.002195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe7e86-8347-44ba-9963-77bb3519b66d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.002637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe7e86-8347-44ba-9963-77bb3519b66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.028262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:36.039771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0cfe7e86-8347-44ba-9963-77bb3519b66d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:36.043016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0cfe7e86-8347-44ba-9963-77bb3519b66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:36.056993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 899/1802] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:36.061312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b375abe-0290-4293-8a34-6dd9c862c8a9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.077271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b375abe-0290-4293-8a34-6dd9c862c8a9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:36.094809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13770233-3b1b-4e95-ae4e-58a324cd9a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:36.096992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13770233-3b1b-4e95-ae4e-58a324cd9a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.097603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13770233-3b1b-4e95-ae4e-58a324cd9a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.098216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13770233-3b1b-4e95-ae4e-58a324cd9a70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.099477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13770233-3b1b-4e95-ae4e-58a324cd9a70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:36.100179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13770233-3b1b-4e95-ae4e-58a324cd9a70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:36.102226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13770233-3b1b-4e95-ae4e-58a324cd9a70 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:36.124825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 904/1803] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:36.129071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe7e86-8347-44ba-9963-77bb3519b66d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.142791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe7e86-8347-44ba-9963-77bb3519b66d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:36.148758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beccb9-86b3-4ba5-b095-85b1d04af2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:36.150023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beccb9-86b3-4ba5-b095-85b1d04af2eb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.150641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beccb9-86b3-4ba5-b095-85b1d04af2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.152179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24beccb9-86b3-4ba5-b095-85b1d04af2eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.153027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24beccb9-86b3-4ba5-b095-85b1d04af2eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:36.156892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24beccb9-86b3-4ba5-b095-85b1d04af2eb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:36.199853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 900/1804] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:36 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:36.200757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13770233-3b1b-4e95-ae4e-58a324cd9a70 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.214053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13770233-3b1b-4e95-ae4e-58a324cd9a70 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:36.292188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85a9fc-5cae-4acc-9cb1-fd589f0991e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:36.293021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a9fc-5cae-4acc-9cb1-fd589f0991e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.293368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a9fc-5cae-4acc-9cb1-fd589f0991e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.293761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a9fc-5cae-4acc-9cb1-fd589f0991e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.295503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85a9fc-5cae-4acc-9cb1-fd589f0991e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:36.297888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85a9fc-5cae-4acc-9cb1-fd589f0991e0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:36.335121 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 905/1805] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:36 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:36.336695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beccb9-86b3-4ba5-b095-85b1d04af2eb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.353672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24beccb9-86b3-4ba5-b095-85b1d04af2eb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:36.372315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c880962-e79e-4caa-81c0-3e46c5c0c909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:36.372953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c880962-e79e-4caa-81c0-3e46c5c0c909 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.373228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c880962-e79e-4caa-81c0-3e46c5c0c909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.376929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c880962-e79e-4caa-81c0-3e46c5c0c909 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.377713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c880962-e79e-4caa-81c0-3e46c5c0c909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:36.379649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c880962-e79e-4caa-81c0-3e46c5c0c909 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:36.394149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 901/1806] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:36.396252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85a9fc-5cae-4acc-9cb1-fd589f0991e0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.411374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85a9fc-5cae-4acc-9cb1-fd589f0991e0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:36.444488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e971226-4103-4d28-b1d8-f8328d04004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:36.445224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971226-4103-4d28-b1d8-f8328d04004b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.445748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971226-4103-4d28-b1d8-f8328d04004b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.446089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971226-4103-4d28-b1d8-f8328d04004b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.451262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e971226-4103-4d28-b1d8-f8328d04004b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:36.452936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e971226-4103-4d28-b1d8-f8328d04004b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:36.468875 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 906/1807] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:36.471165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c880962-e79e-4caa-81c0-3e46c5c0c909 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.485558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c880962-e79e-4caa-81c0-3e46c5c0c909 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:36.493404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3835f9c-49ab-40df-99c4-3b1f129a086d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:36.494087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3835f9c-49ab-40df-99c4-3b1f129a086d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.494365 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3835f9c-49ab-40df-99c4-3b1f129a086d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.494670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3835f9c-49ab-40df-99c4-3b1f129a086d admin admin] PATH_INFO: `/v3/projects/b7410a40fda441e0b3c41b1872c67ab3/users/f774fb0bb7a44efebd65f2f863b493f4/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.518981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3835f9c-49ab-40df-99c4-3b1f129a086d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7410a40fda441e0b3c41b1872c67ab3, user_id=f774fb0bb7a44efebd65f2f863b493f4, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:36.521952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3835f9c-49ab-40df-99c4-3b1f129a086d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:36.536685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 902/1808] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:36 2022] PUT /identity/v3/projects/b7410a40fda441e0b3c41b1872c67ab3/users/f774fb0bb7a44efebd65f2f863b493f4/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:36.550109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ed1e5-0640-4062-a7ea-374b73860f06 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.550394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ed1e5-0640-4062-a7ea-374b73860f06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.550700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ed1e5-0640-4062-a7ea-374b73860f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.585223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6c4ed1e5-0640-4062-a7ea-374b73860f06 None None] MFA Rules not processed for user `f774fb0bb7a44efebd65f2f863b493f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:36.587024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c4ed1e5-0640-4062-a7ea-374b73860f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:36.682975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 907/1809] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:36.687969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3835f9c-49ab-40df-99c4-3b1f129a086d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.791509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3835f9c-49ab-40df-99c4-3b1f129a086d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:36.797638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6e6e1-425a-4a4b-8999-9648fe703397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:36.798701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6e6e1-425a-4a4b-8999-9648fe703397 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.799167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6e6e1-425a-4a4b-8999-9648fe703397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.799754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6e6e1-425a-4a4b-8999-9648fe703397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.801172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6e6e1-425a-4a4b-8999-9648fe703397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:36.803601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6e6e1-425a-4a4b-8999-9648fe703397 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:36.841900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 903/1810] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:36 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:36.843250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ed1e5-0640-4062-a7ea-374b73860f06 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.855491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ed1e5-0640-4062-a7ea-374b73860f06 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:36.861361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:36.920549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0e74e-5100-498d-8c8d-430ebc05c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:36.921189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0e74e-5100-498d-8c8d-430ebc05c1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.921480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0e74e-5100-498d-8c8d-430ebc05c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.921758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0e74e-5100-498d-8c8d-430ebc05c1c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.922418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0e74e-5100-498d-8c8d-430ebc05c1c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:36.924379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0e74e-5100-498d-8c8d-430ebc05c1c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:36.946898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 908/1811] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:36 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:36.950913 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6e6e1-425a-4a4b-8999-9648fe703397 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.961416 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6e6e1-425a-4a4b-8999-9648fe703397 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:36.965952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd2a53-4d9d-4e34-9292-ac76e04c05f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:36.966594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd2a53-4d9d-4e34-9292-ac76e04c05f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:36.966879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd2a53-4d9d-4e34-9292-ac76e04c05f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:36.967151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd2a53-4d9d-4e34-9292-ac76e04c05f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:36.967842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd2a53-4d9d-4e34-9292-ac76e04c05f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:36.969436 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd2a53-4d9d-4e34-9292-ac76e04c05f8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:36.980754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 904/1812] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:36.984773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0e74e-5100-498d-8c8d-430ebc05c1c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:36.996200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0e74e-5100-498d-8c8d-430ebc05c1c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:37.000873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622b5bc-24c1-4889-9f67-c80de8609cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:37.001536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622b5bc-24c1-4889-9f67-c80de8609cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:37.001820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622b5bc-24c1-4889-9f67-c80de8609cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:37.002095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622b5bc-24c1-4889-9f67-c80de8609cd5 admin admin] PATH_INFO: `/v3/projects/95bfcbe831804016971bccb04fc54aee/users/979e4659f1ed4514b830f862eade9329/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:37.017612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622b5bc-24c1-4889-9f67-c80de8609cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95bfcbe831804016971bccb04fc54aee, user_id=979e4659f1ed4514b830f862eade9329, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:37.020420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622b5bc-24c1-4889-9f67-c80de8609cd5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:37.033637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 909/1813] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:36 2022] PUT /identity/v3/projects/95bfcbe831804016971bccb04fc54aee/users/979e4659f1ed4514b830f862eade9329/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:37.037628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd2a53-4d9d-4e34-9292-ac76e04c05f8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:37.048713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd2a53-4d9d-4e34-9292-ac76e04c05f8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:37.129166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da896e53-9d6b-4af7-aa22-f2b0e7aba4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:37.130215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da896e53-9d6b-4af7-aa22-f2b0e7aba4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:37.131786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da896e53-9d6b-4af7-aa22-f2b0e7aba4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:37.132484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da896e53-9d6b-4af7-aa22-f2b0e7aba4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:37.133974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da896e53-9d6b-4af7-aa22-f2b0e7aba4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:37.136839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da896e53-9d6b-4af7-aa22-f2b0e7aba4f9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:37.166569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 905/1814] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:37.171059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622b5bc-24c1-4889-9f67-c80de8609cd5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:37.182753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622b5bc-24c1-4889-9f67-c80de8609cd5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:37.187457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc841b67-a8e2-45be-a1aa-05ff5859e87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:37.188196 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc841b67-a8e2-45be-a1aa-05ff5859e87a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:37.188510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc841b67-a8e2-45be-a1aa-05ff5859e87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:37.188953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc841b67-a8e2-45be-a1aa-05ff5859e87a admin admin] PATH_INFO: `/v3/projects/95bfcbe831804016971bccb04fc54aee/users/979e4659f1ed4514b830f862eade9329/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:37.191319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc841b67-a8e2-45be-a1aa-05ff5859e87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=95bfcbe831804016971bccb04fc54aee, user_id=979e4659f1ed4514b830f862eade9329, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:37.194614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc841b67-a8e2-45be-a1aa-05ff5859e87a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:37.205739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 910/1815] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:37 2022] PUT /identity/v3/projects/95bfcbe831804016971bccb04fc54aee/users/979e4659f1ed4514b830f862eade9329/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:37.216401 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b37d8-97de-475b-984e-0947e4824a18 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:37.216707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b37d8-97de-475b-984e-0947e4824a18 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:37.217099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b37d8-97de-475b-984e-0947e4824a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:37.239797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:37.249608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-be3b37d8-97de-475b-984e-0947e4824a18 None None] MFA Rules not processed for user `979e4659f1ed4514b830f862eade9329`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:37.251443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-be3b37d8-97de-475b-984e-0947e4824a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:37.344548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 906/1816] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:37.365872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc841b67-a8e2-45be-a1aa-05ff5859e87a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:37.372689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc841b67-a8e2-45be-a1aa-05ff5859e87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:37.560003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc841b67-a8e2-45be-a1aa-05ff5859e87a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:37.568993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8d46b-3d31-4740-827b-35762cbde2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:37.569645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8d46b-3d31-4740-827b-35762cbde2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:37.569928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8d46b-3d31-4740-827b-35762cbde2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:37.570525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8d46b-3d31-4740-827b-35762cbde2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:37.588345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8d46b-3d31-4740-827b-35762cbde2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:37.591871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8d46b-3d31-4740-827b-35762cbde2f4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:37.611931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 911/1817] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:58:38.025599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb9615-f5ce-4eca-a6f7-b7cec9a488a2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:38.025599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb9615-f5ce-4eca-a6f7-b7cec9a488a2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:38.025599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb9615-f5ce-4eca-a6f7-b7cec9a488a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:38.177319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-49eb9615-f5ce-4eca-a6f7-b7cec9a488a2 None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:38.178758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-49eb9615-f5ce-4eca-a6f7-b7cec9a488a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:38.364047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 907/1818] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 04:58:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:38.665750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8d46b-3d31-4740-827b-35762cbde2f4 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:38.667271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-04f8d46b-3d31-4740-827b-35762cbde2f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:38.690658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb9615-f5ce-4eca-a6f7-b7cec9a488a2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:38.716041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb9615-f5ce-4eca-a6f7-b7cec9a488a2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:38.729966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762d70e-8c5c-4c54-9ca1-e761da857533 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e4659f1ed4514b830f862eade9329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95bfcbe831804016971bccb04fc54aee', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:38.733143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762d70e-8c5c-4c54-9ca1-e761da857533 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:38.735493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762d70e-8c5c-4c54-9ca1-e761da857533 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:38.735995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762d70e-8c5c-4c54-9ca1-e761da857533 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:38.737351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762d70e-8c5c-4c54-9ca1-e761da857533 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:38.739570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762d70e-8c5c-4c54-9ca1-e761da857533 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:38.744497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762d70e-8c5c-4c54-9ca1-e761da857533 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:38.761053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:38.776713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 908/1819] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:38.781068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762d70e-8c5c-4c54-9ca1-e761da857533 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:38.815073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762d70e-8c5c-4c54-9ca1-e761da857533 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:38.828563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8d46b-3d31-4740-827b-35762cbde2f4 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:38.837147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff063513-2828-4c72-9444-5eb052da0a73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:38.839353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff063513-2828-4c72-9444-5eb052da0a73 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:38.839649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff063513-2828-4c72-9444-5eb052da0a73 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:38.839936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff063513-2828-4c72-9444-5eb052da0a73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:38.843572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e638daa-c239-407b-a160-9f840c38b093 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e4659f1ed4514b830f862eade9329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95bfcbe831804016971bccb04fc54aee', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:38.844293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e638daa-c239-407b-a160-9f840c38b093 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:38.845143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e638daa-c239-407b-a160-9f840c38b093 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:38.846130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e638daa-c239-407b-a160-9f840c38b093 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:38.847461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e638daa-c239-407b-a160-9f840c38b093 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:38.854280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff063513-2828-4c72-9444-5eb052da0a73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:38.855523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e638daa-c239-407b-a160-9f840c38b093 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:38.856172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff063513-2828-4c72-9444-5eb052da0a73 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:38.886731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 912/1820] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 04:58:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 04:58:38.927744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 909/1821] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:38 2022] POST /identity/v3/projects => generated 353 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:38.934465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff063513-2828-4c72-9444-5eb052da0a73 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:38.980269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff063513-2828-4c72-9444-5eb052da0a73 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:39.199793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e4659f1ed4514b830f862eade9329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95bfcbe831804016971bccb04fc54aee', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:39.200812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:39.201306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:39.201784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:39.206340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:39.206895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:39.208907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:39.245200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 913/1822] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:39.325766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e638daa-c239-407b-a160-9f840c38b093 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:39.352837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e638daa-c239-407b-a160-9f840c38b093 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:39.366708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '979e4659f1ed4514b830f862eade9329', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95bfcbe831804016971bccb04fc54aee', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:39.368428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:39.368542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:39.368815 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] PATH_INFO: `/v3/projects/30698bfd238544dbaadb2b1a2f84602d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:39.370154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Authorizing `identity:delete_project(project_id=30698bfd238544dbaadb2b1a2f84602d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:39.372635 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:39.490077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:39.555310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30698bfd238544dbaadb2b1a2f84602d', 'request_id': 'req-cce56b40-2af5-4003-bd97-ff95649ead7d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:39.570324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30698bfd238544dbaadb2b1a2f84602d', 'request_id': 'req-cce56b40-2af5-4003-bd97-ff95649ead7d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:39.583109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30698bfd238544dbaadb2b1a2f84602d', 'request_id': 'req-cce56b40-2af5-4003-bd97-ff95649ead7d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:39.632046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] The token cache is being invalidate because project 30698bfd238544dbaadb2b1a2f84602d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:39.632378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:39.638580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:39.642093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 910/1823] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:39 2022] DELETE /identity/v3/projects/30698bfd238544dbaadb2b1a2f84602d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:40.312490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:40.314469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:40.484332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84015d-4230-4fdc-a72f-1a9ae002d432 tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:40.717193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:40.723110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:40.723435 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:40.723716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] PATH_INFO: `/v3/users/f774fb0bb7a44efebd65f2f863b493f4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:40.725363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f774fb0bb7a44efebd65f2f863b493f4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:40.735796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:40.844618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:40.846710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:40.848914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:40.881518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce56b40-2af5-4003-bd97-ff95649ead7d tempest-VolumeQuotaClassesTest-1805800896 tempest-VolumeQuotaClassesTest-1805800896-project-admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:40.910169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:40.911981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:40.912843 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:40.913279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] PATH_INFO: `/v3/users/ccde0065abf8481b969790329cc4ca77` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:40.920764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccde0065abf8481b969790329cc4ca77)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:40.931971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:40.950825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f774fb0bb7a44efebd65f2f863b493f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:40.975487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f774fb0bb7a44efebd65f2f863b493f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:40.977823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f774fb0bb7a44efebd65f2f863b493f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:40.980087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f774fb0bb7a44efebd65f2f863b493f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:41.035361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71afeaa7-ae47-46f7-b075-e3da8da57d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f774fb0bb7a44efebd65f2f863b493f4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:41.078315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 914/1824] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:40 2022] DELETE /identity/v3/users/f774fb0bb7a44efebd65f2f863b493f4 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:58:41.083015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82337f22-317e-4fd0-a11b-1905776f39fa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:41.083328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82337f22-317e-4fd0-a11b-1905776f39fa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:41.083603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82337f22-317e-4fd0-a11b-1905776f39fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:41.109672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccde0065abf8481b969790329cc4ca77'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:41.128045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccde0065abf8481b969790329cc4ca77'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:41.128825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccde0065abf8481b969790329cc4ca77'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:41.129465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccde0065abf8481b969790329cc4ca77'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:41.148480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-82337f22-317e-4fd0-a11b-1905776f39fa None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:41.150485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-82337f22-317e-4fd0-a11b-1905776f39fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:41.157529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-69efc4f6-75f0-4866-82f1-a27768619ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccde0065abf8481b969790329cc4ca77'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:41.190500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 911/1825] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:40 2022] DELETE /identity/v3/users/ccde0065abf8481b969790329cc4ca77 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:41.195980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6006d-f158-4f1f-be42-b62f257a15f0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:41.195980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6006d-f158-4f1f-be42-b62f257a15f0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:41.196724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6006d-f158-4f1f-be42-b62f257a15f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:41.237812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0ad6006d-f158-4f1f-be42-b62f257a15f0 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:41.239186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0ad6006d-f158-4f1f-be42-b62f257a15f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:41.308013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 915/1826] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:41.384082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82337f22-317e-4fd0-a11b-1905776f39fa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:41.386558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-82337f22-317e-4fd0-a11b-1905776f39fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:41.393375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 912/1827] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:41.447326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6006d-f158-4f1f-be42-b62f257a15f0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:41.449230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0ad6006d-f158-4f1f-be42-b62f257a15f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:41.465629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82337f22-317e-4fd0-a11b-1905776f39fa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:41.496354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:41.541178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6006d-f158-4f1f-be42-b62f257a15f0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:41.548440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743a232-0a59-423e-aa84-fccb61de0456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:41.549895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743a232-0a59-423e-aa84-fccb61de0456 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:41.559430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743a232-0a59-423e-aa84-fccb61de0456 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:41.559713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743a232-0a59-423e-aa84-fccb61de0456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:41.564665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ceb3f0-ae11-405c-9549-74da120e6f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:41.565864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ceb3f0-ae11-405c-9549-74da120e6f7c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:41.565975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ceb3f0-ae11-405c-9549-74da120e6f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:41.566167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ceb3f0-ae11-405c-9549-74da120e6f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:41.579716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743a232-0a59-423e-aa84-fccb61de0456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:41.582351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:41.585029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743a232-0a59-423e-aa84-fccb61de0456 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:41.592924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ceb3f0-ae11-405c-9549-74da120e6f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:41.598226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ceb3f0-ae11-405c-9549-74da120e6f7c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:41.615137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 913/1828] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:41.620310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 916/1829] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:58:43.076100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743a232-0a59-423e-aa84-fccb61de0456 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:43.078631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d743a232-0a59-423e-aa84-fccb61de0456 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:43.116775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ceb3f0-ae11-405c-9549-74da120e6f7c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:43.118210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743a232-0a59-423e-aa84-fccb61de0456 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:43.119888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d2ceb3f0-ae11-405c-9549-74da120e6f7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:43.127945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:43.129270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:43.129980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:43.130780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] PATH_INFO: `/v3/projects/b7410a40fda441e0b3c41b1872c67ab3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:43.134868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7410a40fda441e0b3c41b1872c67ab3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:43.136878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:43.142560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ceb3f0-ae11-405c-9549-74da120e6f7c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:43.178665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:43.180498 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:43.180780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:43.181775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] PATH_INFO: `/v3/projects/2ffc60ac499b456f8ec93a9d57c14df3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:43.184172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffc60ac499b456f8ec93a9d57c14df3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:43.188045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:43.230151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7410a40fda441e0b3c41b1872c67ab3', 'request_id': 'req-f57dad21-1659-42a3-8b41-e0e32d1a6083'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.261740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7410a40fda441e0b3c41b1872c67ab3', 'request_id': 'req-f57dad21-1659-42a3-8b41-e0e32d1a6083'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.271630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7410a40fda441e0b3c41b1872c67ab3', 'request_id': 'req-f57dad21-1659-42a3-8b41-e0e32d1a6083'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.282428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffc60ac499b456f8ec93a9d57c14df3', 'request_id': 'req-9beee773-f437-4c6e-a25f-e10025e4d42a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.298064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffc60ac499b456f8ec93a9d57c14df3', 'request_id': 'req-9beee773-f437-4c6e-a25f-e10025e4d42a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.301935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] The token cache is being invalidate because project b7410a40fda441e0b3c41b1872c67ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:43.302429 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.303099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.307409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 914/1830] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:43 2022] DELETE /identity/v3/projects/b7410a40fda441e0b3c41b1872c67ab3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:43.311346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffc60ac499b456f8ec93a9d57c14df3', 'request_id': 'req-9beee773-f437-4c6e-a25f-e10025e4d42a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.311936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:43.313994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:43.331779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] The token cache is being invalidate because project 2ffc60ac499b456f8ec93a9d57c14df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:43.332078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.333242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.345180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 917/1831] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:43 2022] DELETE /identity/v3/projects/2ffc60ac499b456f8ec93a9d57c14df3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:43.346230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:43.347691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:43.410172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dad21-1659-42a3-8b41-e0e32d1a6083 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:43.437476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beee773-f437-4c6e-a25f-e10025e4d42a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:43.442388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:43.443139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:43.443490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:43.443828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] PATH_INFO: `/v3/users/2a5e95d693a84ebcaae721f838403c92` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:43.445351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5e95d693a84ebcaae721f838403c92)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:43.447491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:43.499033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:43.499844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:43.500226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:43.500503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] PATH_INFO: `/v3/users/979e4659f1ed4514b830f862eade9329` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:43.502954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=979e4659f1ed4514b830f862eade9329)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:43.505407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:43.509781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:43.515107 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5e95d693a84ebcaae721f838403c92'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.524691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5e95d693a84ebcaae721f838403c92'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.525652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5e95d693a84ebcaae721f838403c92'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.526468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5e95d693a84ebcaae721f838403c92'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.540518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5e95d693a84ebcaae721f838403c92'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.545557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:43.560590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 918/1832] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:43 2022] DELETE /identity/v3/users/2a5e95d693a84ebcaae721f838403c92 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:43.578426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '979e4659f1ed4514b830f862eade9329'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.595109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '979e4659f1ed4514b830f862eade9329'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.595536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '979e4659f1ed4514b830f862eade9329'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.596527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '979e4659f1ed4514b830f862eade9329'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.623615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '979e4659f1ed4514b830f862eade9329'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:43.657186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 915/1833] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:43 2022] DELETE /identity/v3/users/979e4659f1ed4514b830f862eade9329 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:44.109354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:44.111516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:44.114394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:44.116043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:44.235352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8110e-4cf4-415e-a1b3-532a00db39a2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:44.316640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:44.317564 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.318007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.318475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] PATH_INFO: `/v3/projects/c91f5c0afece492b83f02cfbfd2bb09c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:44.321283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c91f5c0afece492b83f02cfbfd2bb09c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:44.322431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c2b01-080b-4d7a-8337-bd4a33f1ae4d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:44.323931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:44.344286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:44.344286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.344286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.344286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] PATH_INFO: `/v3/projects/95bfcbe831804016971bccb04fc54aee` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:44.344651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95bfcbe831804016971bccb04fc54aee)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:44.345135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:44.394559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c91f5c0afece492b83f02cfbfd2bb09c', 'request_id': 'req-c2ccc105-2c65-4daa-ba32-481ba8b3d898'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.410995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c91f5c0afece492b83f02cfbfd2bb09c', 'request_id': 'req-c2ccc105-2c65-4daa-ba32-481ba8b3d898'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.421173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95bfcbe831804016971bccb04fc54aee', 'request_id': 'req-612b38e2-a054-4755-a500-55230aeff712'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.423505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c91f5c0afece492b83f02cfbfd2bb09c', 'request_id': 'req-c2ccc105-2c65-4daa-ba32-481ba8b3d898'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.434690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95bfcbe831804016971bccb04fc54aee', 'request_id': 'req-612b38e2-a054-4755-a500-55230aeff712'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.442862 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] The token cache is being invalidate because project c91f5c0afece492b83f02cfbfd2bb09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:44.443329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.444141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c2ccc105-2c65-4daa-ba32-481ba8b3d898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.444702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95bfcbe831804016971bccb04fc54aee', 'request_id': 'req-612b38e2-a054-4755-a500-55230aeff712'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.447239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 919/1834] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:44 2022] DELETE /identity/v3/projects/c91f5c0afece492b83f02cfbfd2bb09c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:44.463268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c403e69-1e09-4f39-bfc5-8637493d409e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.463468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] The token cache is being invalidate because project 95bfcbe831804016971bccb04fc54aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:44.463754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.464475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-612b38e2-a054-4755-a500-55230aeff712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:44.466730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c403e69-1e09-4f39-bfc5-8637493d409e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.467251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c403e69-1e09-4f39-bfc5-8637493d409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:44.471010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 916/1835] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:44 2022] DELETE /identity/v3/projects/95bfcbe831804016971bccb04fc54aee => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:44.472546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27a240-46d2-4187-9d12-fef7f9e37c7f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.472829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27a240-46d2-4187-9d12-fef7f9e37c7f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.473103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27a240-46d2-4187-9d12-fef7f9e37c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:44.499074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dd27a240-46d2-4187-9d12-fef7f9e37c7f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:44.500321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dd27a240-46d2-4187-9d12-fef7f9e37c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:44.508185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:44.518362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4c403e69-1e09-4f39-bfc5-8637493d409e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:44.526769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4c403e69-1e09-4f39-bfc5-8637493d409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:44.607582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:44.647322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 917/1836] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:44.649135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab2d2-1488-442b-9a5f-745d68b5f639 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.649418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab2d2-1488-442b-9a5f-745d68b5f639 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.649694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab2d2-1488-442b-9a5f-745d68b5f639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:44.696438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6ceab2d2-1488-442b-9a5f-745d68b5f639 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:44.698241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6ceab2d2-1488-442b-9a5f-745d68b5f639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:44.716108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 918/1837] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:44.716108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d833ad0-b911-431d-b77e-91f81f0c8b5f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.716108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d833ad0-b911-431d-b77e-91f81f0c8b5f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.716108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d833ad0-b911-431d-b77e-91f81f0c8b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:44.744749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5d833ad0-b911-431d-b77e-91f81f0c8b5f None None] MFA Rules not processed for user `426be56424504e958402d785a068d2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:44.746410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5d833ad0-b911-431d-b77e-91f81f0c8b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:44.785480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 920/1838] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:44.790412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c403e69-1e09-4f39-bfc5-8637493d409e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:44.819280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c403e69-1e09-4f39-bfc5-8637493d409e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:44.847538 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17514d98-ea1d-42bc-a7a0-3750cacefdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:44.848330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17514d98-ea1d-42bc-a7a0-3750cacefdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.848620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17514d98-ea1d-42bc-a7a0-3750cacefdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.848903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17514d98-ea1d-42bc-a7a0-3750cacefdc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:44.849631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17514d98-ea1d-42bc-a7a0-3750cacefdc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:44.849943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17514d98-ea1d-42bc-a7a0-3750cacefdc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:44.868478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17514d98-ea1d-42bc-a7a0-3750cacefdc3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:44.870794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 919/1839] 10.222.0.49 () {58 vars in 1015 bytes} [Sat Jul 30 04:58:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:44.873521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e5a9b-de91-4be7-9481-ed2617b02f4d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.874791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e5a9b-de91-4be7-9481-ed2617b02f4d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.876441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e5a9b-de91-4be7-9481-ed2617b02f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:44.895579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 921/1840] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:44.895579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17514d98-ea1d-42bc-a7a0-3750cacefdc3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:44.908748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17514d98-ea1d-42bc-a7a0-3750cacefdc3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:44.927922 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c842862-ddd0-401e-ae4e-64aa11c9be07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:44.928491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c842862-ddd0-401e-ae4e-64aa11c9be07 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.928775 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c842862-ddd0-401e-ae4e-64aa11c9be07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.929047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c842862-ddd0-401e-ae4e-64aa11c9be07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:44.929769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c842862-ddd0-401e-ae4e-64aa11c9be07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:44.931735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c842862-ddd0-401e-ae4e-64aa11c9be07 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:44.945350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-eb8e5a9b-de91-4be7-9481-ed2617b02f4d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:44.947660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eb8e5a9b-de91-4be7-9481-ed2617b02f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:44.959333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 920/1841] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:44.964999 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e5a9b-de91-4be7-9481-ed2617b02f4d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:44.967761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eb8e5a9b-de91-4be7-9481-ed2617b02f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:44.985073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e5a9b-de91-4be7-9481-ed2617b02f4d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:44.990133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6f5f-25f2-4de1-aefb-4faa6534fb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:44.991093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6f5f-25f2-4de1-aefb-4faa6534fb1d service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:44.991483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6f5f-25f2-4de1-aefb-4faa6534fb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:44.992211 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6f5f-25f2-4de1-aefb-4faa6534fb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.028824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e6f5f-25f2-4de1-aefb-4faa6534fb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.031527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e6f5f-25f2-4de1-aefb-4faa6534fb1d service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.046663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 922/1842] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:44 2022] POST /identity/v3/projects => generated 409 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:45.046663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c842862-ddd0-401e-ae4e-64aa11c9be07 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.057043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 921/1843] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:45.058074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6f5f-25f2-4de1-aefb-4faa6534fb1d service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.073625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6f5f-25f2-4de1-aefb-4faa6534fb1d service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.183875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.184613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.184786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.185849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.186322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:45.187227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.190168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.191267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c842862-ddd0-401e-ae4e-64aa11c9be07 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.199472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea34df-4b2f-4583-a1c2-e553ef6c5984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.201050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea34df-4b2f-4583-a1c2-e553ef6c5984 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.201700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea34df-4b2f-4583-a1c2-e553ef6c5984 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.202173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea34df-4b2f-4583-a1c2-e553ef6c5984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.210225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 922/1844] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:45.211995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.217748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea34df-4b2f-4583-a1c2-e553ef6c5984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.218426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:45.220207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea34df-4b2f-4583-a1c2-e553ef6c5984 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.233641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 923/1845] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:45.235360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea34df-4b2f-4583-a1c2-e553ef6c5984 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.242971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e5690-6978-4b5a-9c2d-bafca353e346 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.248700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20441922-a721-4016-83cb-e9fb563bc506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.249265 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea34df-4b2f-4583-a1c2-e553ef6c5984 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.249816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20441922-a721-4016-83cb-e9fb563bc506 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.250015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20441922-a721-4016-83cb-e9fb563bc506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.250293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20441922-a721-4016-83cb-e9fb563bc506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.254552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ac6a8-15fe-4686-9eb7-8e7dde9550d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.255440 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ac6a8-15fe-4686-9eb7-8e7dde9550d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.255878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ac6a8-15fe-4686-9eb7-8e7dde9550d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.256310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ac6a8-15fe-4686-9eb7-8e7dde9550d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.261950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20441922-a721-4016-83cb-e9fb563bc506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.264089 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20441922-a721-4016-83cb-e9fb563bc506 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.267634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ac6a8-15fe-4686-9eb7-8e7dde9550d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.269723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ac6a8-15fe-4686-9eb7-8e7dde9550d3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.275265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 923/1846] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:45.276441 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20441922-a721-4016-83cb-e9fb563bc506 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.296992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20441922-a721-4016-83cb-e9fb563bc506 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.302652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f865c-bc66-446c-965e-bb97de9757e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.303152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 924/1847] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:45.303652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f865c-bc66-446c-965e-bb97de9757e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.305175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f865c-bc66-446c-965e-bb97de9757e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.305361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ac6a8-15fe-4686-9eb7-8e7dde9550d3 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.305598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f865c-bc66-446c-965e-bb97de9757e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.320069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f865c-bc66-446c-965e-bb97de9757e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.322730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f865c-bc66-446c-965e-bb97de9757e8 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.331633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ac6a8-15fe-4686-9eb7-8e7dde9550d3 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.339200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 924/1848] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:45.340244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f865c-bc66-446c-965e-bb97de9757e8 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.371882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e36d2-f4b4-46fc-8770-112384730483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.372961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e36d2-f4b4-46fc-8770-112384730483 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.373577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e36d2-f4b4-46fc-8770-112384730483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.374340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e36d2-f4b4-46fc-8770-112384730483 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.375339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e36d2-f4b4-46fc-8770-112384730483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.378500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:45.382332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e36d2-f4b4-46fc-8770-112384730483 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.393336 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f865c-bc66-446c-965e-bb97de9757e8 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.410108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aecc55-95ed-4b22-a118-3d0ee0655f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.410785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aecc55-95ed-4b22-a118-3d0ee0655f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.411068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aecc55-95ed-4b22-a118-3d0ee0655f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.411347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aecc55-95ed-4b22-a118-3d0ee0655f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.412955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aecc55-95ed-4b22-a118-3d0ee0655f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.417949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aecc55-95ed-4b22-a118-3d0ee0655f5b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.434732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 925/1849] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:45 2022] POST /identity/v3/users => generated 305 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:45.437152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e36d2-f4b4-46fc-8770-112384730483 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.464871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e36d2-f4b4-46fc-8770-112384730483 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.472530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca95dd-89fb-4198-9afa-2cdb38c8d175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.473330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca95dd-89fb-4198-9afa-2cdb38c8d175 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.473724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca95dd-89fb-4198-9afa-2cdb38c8d175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.474103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca95dd-89fb-4198-9afa-2cdb38c8d175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.474959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca95dd-89fb-4198-9afa-2cdb38c8d175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.477562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca95dd-89fb-4198-9afa-2cdb38c8d175 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.488434 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 925/1850] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:45 2022] POST /identity/v3/projects => generated 361 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:45.496606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 926/1851] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:45.496804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aecc55-95ed-4b22-a118-3d0ee0655f5b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.497857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca95dd-89fb-4198-9afa-2cdb38c8d175 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.524242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca95dd-89fb-4198-9afa-2cdb38c8d175 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.605110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aecc55-95ed-4b22-a118-3d0ee0655f5b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.613192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889b66d-d954-46ae-a27c-4d709542bd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.614081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889b66d-d954-46ae-a27c-4d709542bd8b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.614586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889b66d-d954-46ae-a27c-4d709542bd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.615014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889b66d-d954-46ae-a27c-4d709542bd8b admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.615899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889b66d-d954-46ae-a27c-4d709542bd8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.618650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889b66d-d954-46ae-a27c-4d709542bd8b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.632066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e9852-2559-4c0c-be50-1c1330baebd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.641491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e9852-2559-4c0c-be50-1c1330baebd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.644378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e9852-2559-4c0c-be50-1c1330baebd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.644810 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e9852-2559-4c0c-be50-1c1330baebd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.648067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e9852-2559-4c0c-be50-1c1330baebd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.650353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570e9852-2559-4c0c-be50-1c1330baebd7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.650990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 926/1852] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:45 2022] POST /identity/v3/users => generated 281 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:45.654885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889b66d-d954-46ae-a27c-4d709542bd8b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.663109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:45.668290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889b66d-d954-46ae-a27c-4d709542bd8b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.677199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a3a9-516d-422a-b122-1a94413f1f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.677900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a3a9-516d-422a-b122-1a94413f1f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.678181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a3a9-516d-422a-b122-1a94413f1f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.678469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528a3a9-516d-422a-b122-1a94413f1f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.679724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528a3a9-516d-422a-b122-1a94413f1f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.681531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528a3a9-516d-422a-b122-1a94413f1f2f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.684022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 927/1853] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:45.688918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e9852-2559-4c0c-be50-1c1330baebd7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.706649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 927/1854] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:45.710148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a3a9-516d-422a-b122-1a94413f1f2f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.711702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e9852-2559-4c0c-be50-1c1330baebd7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.730161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12788d0-f306-4a8c-b763-048bb2f35dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.731055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12788d0-f306-4a8c-b763-048bb2f35dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.731453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12788d0-f306-4a8c-b763-048bb2f35dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.731843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12788d0-f306-4a8c-b763-048bb2f35dd4 admin admin] PATH_INFO: `/v3/projects/1c7feec501a9450ba2ed07ca24b01aa2/users/8394d05ebe404329ae33220c5d74d4e8/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.738872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528a3a9-516d-422a-b122-1a94413f1f2f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.752161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12788d0-f306-4a8c-b763-048bb2f35dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7feec501a9450ba2ed07ca24b01aa2, user_id=8394d05ebe404329ae33220c5d74d4e8, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.753004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d9923-46d1-44cc-83cc-9cd2442a5e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:45.754991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d9923-46d1-44cc-83cc-9cd2442a5e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.755275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d9923-46d1-44cc-83cc-9cd2442a5e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.755544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d9923-46d1-44cc-83cc-9cd2442a5e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.755713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12788d0-f306-4a8c-b763-048bb2f35dd4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.758104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d9923-46d1-44cc-83cc-9cd2442a5e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:45.762397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d9923-46d1-44cc-83cc-9cd2442a5e9e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:45.782272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 928/1855] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:45 2022] PUT /identity/v3/projects/1c7feec501a9450ba2ed07ca24b01aa2/users/8394d05ebe404329ae33220c5d74d4e8/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:45.796551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e1210-8843-46a9-8e2c-4782b4d8921c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:45.797220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e1210-8843-46a9-8e2c-4782b4d8921c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:45.797716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e1210-8843-46a9-8e2c-4782b4d8921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:45.798593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 928/1856] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:45.802389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d9923-46d1-44cc-83cc-9cd2442a5e9e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:45.817894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d9923-46d1-44cc-83cc-9cd2442a5e9e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:45.913255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5b6e1210-8843-46a9-8e2c-4782b4d8921c None None] MFA Rules not processed for user `8394d05ebe404329ae33220c5d74d4e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:45.918409 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5b6e1210-8843-46a9-8e2c-4782b4d8921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:46.077111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccce8c4-ce22-48aa-9446-e22a12f565b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:46.078309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccce8c4-ce22-48aa-9446-e22a12f565b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:46.078854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccce8c4-ce22-48aa-9446-e22a12f565b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:46.079237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccce8c4-ce22-48aa-9446-e22a12f565b2 admin admin] PATH_INFO: `/v3/projects/eb98a151879f44178106e60d99be0f10/users/456dd164928e49869ea3d95ec07f01b7/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:46.130694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccce8c4-ce22-48aa-9446-e22a12f565b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb98a151879f44178106e60d99be0f10, user_id=456dd164928e49869ea3d95ec07f01b7, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:46.134375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccce8c4-ce22-48aa-9446-e22a12f565b2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:46.170271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 929/1857] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:45 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:46.172976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e1210-8843-46a9-8e2c-4782b4d8921c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:46.208651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 929/1858] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:45 2022] PUT /identity/v3/projects/eb98a151879f44178106e60d99be0f10/users/456dd164928e49869ea3d95ec07f01b7/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:58:46.222109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea487549-5151-4bbf-b1c5-b9fab004df37 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:46.222398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea487549-5151-4bbf-b1c5-b9fab004df37 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:46.222716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea487549-5151-4bbf-b1c5-b9fab004df37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:46.235270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e1210-8843-46a9-8e2c-4782b4d8921c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:46.302637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ea487549-5151-4bbf-b1c5-b9fab004df37 None None] MFA Rules not processed for user `456dd164928e49869ea3d95ec07f01b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:46.304786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ea487549-5151-4bbf-b1c5-b9fab004df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:46.356288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:46.488543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cdb94-28c9-4745-bdc7-c50a9b859ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:46.490129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cdb94-28c9-4745-bdc7-c50a9b859ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:46.490637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cdb94-28c9-4745-bdc7-c50a9b859ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:46.494886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cdb94-28c9-4745-bdc7-c50a9b859ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:46.494886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cdb94-28c9-4745-bdc7-c50a9b859ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:46.495209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cdb94-28c9-4745-bdc7-c50a9b859ea7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:46.495302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 930/1859] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:46.660472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 930/1860] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:46 2022] POST /identity/v3/projects => generated 409 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:46.660596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea487549-5151-4bbf-b1c5-b9fab004df37 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:46.681882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea487549-5151-4bbf-b1c5-b9fab004df37 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:46.689041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cdb94-28c9-4745-bdc7-c50a9b859ea7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:46.691276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f21cdb94-28c9-4745-bdc7-c50a9b859ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:46.857234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:46.874909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cdb94-28c9-4745-bdc7-c50a9b859ea7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:46.888700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da57d20-9468-441d-87e8-721c70557fd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:46.891568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da57d20-9468-441d-87e8-721c70557fd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:46.891941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da57d20-9468-441d-87e8-721c70557fd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:46.892319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da57d20-9468-441d-87e8-721c70557fd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:46.896382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e85727-9ce6-4dbb-9f99-1981f6c99ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:46.901467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e85727-9ce6-4dbb-9f99-1981f6c99ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:46.902919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e85727-9ce6-4dbb-9f99-1981f6c99ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:46.903711 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e85727-9ce6-4dbb-9f99-1981f6c99ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:46.906109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e85727-9ce6-4dbb-9f99-1981f6c99ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:46.911786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e85727-9ce6-4dbb-9f99-1981f6c99ad8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:46.978617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 931/1861] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:46 2022] POST /identity/v3/users => generated 311 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:46.979650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e85727-9ce6-4dbb-9f99-1981f6c99ad8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:47.012674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e85727-9ce6-4dbb-9f99-1981f6c99ad8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:47.034803 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a394c-9448-4426-8907-b9c13b99d82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:47.044745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a394c-9448-4426-8907-b9c13b99d82c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:47.045726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a394c-9448-4426-8907-b9c13b99d82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:47.046034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a394c-9448-4426-8907-b9c13b99d82c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:47.047703 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a394c-9448-4426-8907-b9c13b99d82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:47.050158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a394c-9448-4426-8907-b9c13b99d82c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:47.084679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 932/1862] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:47.090410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a394c-9448-4426-8907-b9c13b99d82c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:47.091895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da57d20-9468-441d-87e8-721c70557fd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:47.093509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da57d20-9468-441d-87e8-721c70557fd0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:47.121110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a394c-9448-4426-8907-b9c13b99d82c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:47.122172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 931/1863] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:58:47.140462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da57d20-9468-441d-87e8-721c70557fd0 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:47.150634 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33629e96-715c-42e2-8e3c-91f1a5ec783d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:47.151329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33629e96-715c-42e2-8e3c-91f1a5ec783d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:47.151652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33629e96-715c-42e2-8e3c-91f1a5ec783d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:47.151880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33629e96-715c-42e2-8e3c-91f1a5ec783d admin admin] PATH_INFO: `/v3/projects/da6396e6f3be4e829c4641baae5eaab8/users/22dc72956b344c0b833a34c96c8cbfe2/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:47.171384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da57d20-9468-441d-87e8-721c70557fd0 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:47.187610 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:47.188348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:47.189746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:47.192710 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] PATH_INFO: `/v3/users/456dd164928e49869ea3d95ec07f01b7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:47.196309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=456dd164928e49869ea3d95ec07f01b7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:47.202636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33629e96-715c-42e2-8e3c-91f1a5ec783d admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6396e6f3be4e829c4641baae5eaab8, user_id=22dc72956b344c0b833a34c96c8cbfe2, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:47.204365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:47.213756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33629e96-715c-42e2-8e3c-91f1a5ec783d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:47.241436 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 933/1864] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:47 2022] PUT /identity/v3/projects/da6396e6f3be4e829c4641baae5eaab8/users/22dc72956b344c0b833a34c96c8cbfe2/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:47.246255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33629e96-715c-42e2-8e3c-91f1a5ec783d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:47.318213 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '456dd164928e49869ea3d95ec07f01b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:47.338181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '456dd164928e49869ea3d95ec07f01b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:47.340273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '456dd164928e49869ea3d95ec07f01b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:47.341141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '456dd164928e49869ea3d95ec07f01b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:47.372179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae883440-5ec8-49c2-84d8-4a5df809d5a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '456dd164928e49869ea3d95ec07f01b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:47.410486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 932/1865] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:47 2022] DELETE /identity/v3/users/456dd164928e49869ea3d95ec07f01b7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:47.410486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f941e9-0920-4f62-8a4d-b8634f458152 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:47.410486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f941e9-0920-4f62-8a4d-b8634f458152 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:47.410486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f941e9-0920-4f62-8a4d-b8634f458152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:47.410985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33629e96-715c-42e2-8e3c-91f1a5ec783d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:47.451545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f8f941e9-0920-4f62-8a4d-b8634f458152 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:47.452920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f8f941e9-0920-4f62-8a4d-b8634f458152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:47.471905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:47.556743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 933/1866] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:47.557419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3418aec-bdb4-415b-9fbe-e9d6d7167da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:47.557763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3418aec-bdb4-415b-9fbe-e9d6d7167da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:47.558048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3418aec-bdb4-415b-9fbe-e9d6d7167da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:47.558329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3418aec-bdb4-415b-9fbe-e9d6d7167da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:47.559064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3418aec-bdb4-415b-9fbe-e9d6d7167da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:47.560954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3418aec-bdb4-415b-9fbe-e9d6d7167da8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:47.561608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ccd7e-7482-4e4a-9bd1-fc34bba4f284 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:47.562278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ccd7e-7482-4e4a-9bd1-fc34bba4f284 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:47.562805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ccd7e-7482-4e4a-9bd1-fc34bba4f284 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:47.565283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 934/1867] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 04:58:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:47.577998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 934/1868] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:47 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:47.582194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ccd7e-7482-4e4a-9bd1-fc34bba4f284 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:47.586502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a29ccd7e-7482-4e4a-9bd1-fc34bba4f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:47.606251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc394a-877d-40f6-a960-a08a1623eba4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:47.606624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc394a-877d-40f6-a960-a08a1623eba4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:47.607129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc394a-877d-40f6-a960-a08a1623eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:47.631990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ccd7e-7482-4e4a-9bd1-fc34bba4f284 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:47.654352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c400c2-3837-45f5-9a8f-c4edf36e2983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:47.655729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c400c2-3837-45f5-9a8f-c4edf36e2983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:47.657460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c400c2-3837-45f5-9a8f-c4edf36e2983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:47.658347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c400c2-3837-45f5-9a8f-c4edf36e2983 admin admin] PATH_INFO: `/v3/projects/da6396e6f3be4e829c4641baae5eaab8/users/22dc72956b344c0b833a34c96c8cbfe2/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:47.672610 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c400c2-3837-45f5-9a8f-c4edf36e2983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6396e6f3be4e829c4641baae5eaab8, user_id=22dc72956b344c0b833a34c96c8cbfe2, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:47.684152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-77bc394a-877d-40f6-a960-a08a1623eba4 None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:47.689234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c400c2-3837-45f5-9a8f-c4edf36e2983 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:47.692242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-77bc394a-877d-40f6-a960-a08a1623eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:47.708716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:47.719565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 935/1869] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:47 2022] PUT /identity/v3/projects/da6396e6f3be4e829c4641baae5eaab8/users/22dc72956b344c0b833a34c96c8cbfe2/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:58:47.731511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6fb2e-ab3f-4476-8815-5ebdfc58e270 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:47.731902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6fb2e-ab3f-4476-8815-5ebdfc58e270 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:47.732279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6fb2e-ab3f-4476-8815-5ebdfc58e270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:47.803638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-55e6fb2e-ab3f-4476-8815-5ebdfc58e270 None None] MFA Rules not processed for user `22dc72956b344c0b833a34c96c8cbfe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:47.806430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55e6fb2e-ab3f-4476-8815-5ebdfc58e270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:47.861884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 935/1870] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 04:58:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:58:47.863368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc394a-877d-40f6-a960-a08a1623eba4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:47.865779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-77bc394a-877d-40f6-a960-a08a1623eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:47.957007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 936/1871] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:47 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:58:47.957404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6fb2e-ab3f-4476-8815-5ebdfc58e270 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:47.958778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55e6fb2e-ab3f-4476-8815-5ebdfc58e270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:47.970228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc394a-877d-40f6-a960-a08a1623eba4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:48.046950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a827e-af7e-4907-9eb7-ad4cf4aae3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:48.047585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a827e-af7e-4907-9eb7-ad4cf4aae3dc service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:48.047859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a827e-af7e-4907-9eb7-ad4cf4aae3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:48.048501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a827e-af7e-4907-9eb7-ad4cf4aae3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:48.072978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:48.073462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6fb2e-ab3f-4476-8815-5ebdfc58e270 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:48.083100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0be75-0e8a-48dc-8132-f7c7d542d4f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:48.083745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0be75-0e8a-48dc-8132-f7c7d542d4f1 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:48.084030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0be75-0e8a-48dc-8132-f7c7d542d4f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:48.084308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0be75-0e8a-48dc-8132-f7c7d542d4f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:48.096554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0be75-0e8a-48dc-8132-f7c7d542d4f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:48.098552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0be75-0e8a-48dc-8132-f7c7d542d4f1 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:48.122992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 937/1872] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:48.125932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0be75-0e8a-48dc-8132-f7c7d542d4f1 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:48.129616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-22b0be75-0e8a-48dc-8132-f7c7d542d4f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:48.131955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a827e-af7e-4907-9eb7-ad4cf4aae3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:48.134321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a827e-af7e-4907-9eb7-ad4cf4aae3dc service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:48.155582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 936/1873] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:48.235859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0be75-0e8a-48dc-8132-f7c7d542d4f1 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:48.240573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4010e-6fb2-4a51-9e20-5688588829d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:48.240910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4010e-6fb2-4a51-9e20-5688588829d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:48.241296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4010e-6fb2-4a51-9e20-5688588829d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:48.241992 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4010e-6fb2-4a51-9e20-5688588829d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:48.261444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4010e-6fb2-4a51-9e20-5688588829d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:48.263465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4010e-6fb2-4a51-9e20-5688588829d2 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:48.293068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 938/1874] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:48 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:48.569012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a827e-af7e-4907-9eb7-ad4cf4aae3dc service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:48.571141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-128a827e-af7e-4907-9eb7-ad4cf4aae3dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:48.583569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a827e-af7e-4907-9eb7-ad4cf4aae3dc service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:48.587329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da628fcb-4603-44ae-a198-883733080524 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:48.587945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da628fcb-4603-44ae-a198-883733080524 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:48.588832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da628fcb-4603-44ae-a198-883733080524 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:48.589108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da628fcb-4603-44ae-a198-883733080524 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:48.590868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-da628fcb-4603-44ae-a198-883733080524 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:48.656162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da628fcb-4603-44ae-a198-883733080524 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:48.657654 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da628fcb-4603-44ae-a198-883733080524 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:48.668007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 937/1875] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:49.553182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928d573-63bf-407d-87b4-728b57c999ff None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:49.554599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928d573-63bf-407d-87b4-728b57c999ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:49.555373 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928d573-63bf-407d-87b4-728b57c999ff None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:49.557239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 939/1876] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 04:58:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:58:49.584735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684de5-7d9a-4698-8e6e-0219ce968d07 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:49.586862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684de5-7d9a-4698-8e6e-0219ce968d07 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:49.586862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44684de5-7d9a-4698-8e6e-0219ce968d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:49.620803 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-44684de5-7d9a-4698-8e6e-0219ce968d07 None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:49.623267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-44684de5-7d9a-4698-8e6e-0219ce968d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:49.638190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 938/1877] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 04:58:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:49.672810 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928d573-63bf-407d-87b4-728b57c999ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:49.675297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5928d573-63bf-407d-87b4-728b57c999ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:49.695639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684de5-7d9a-4698-8e6e-0219ce968d07 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:49.702473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928d573-63bf-407d-87b4-728b57c999ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:49.708582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44684de5-7d9a-4698-8e6e-0219ce968d07 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:49.709305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:49.710228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:49.711070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:49.711343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] PATH_INFO: `/v3/projects/eb98a151879f44178106e60d99be0f10` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:49.713000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb98a151879f44178106e60d99be0f10)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:49.715529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:49.723474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd0a3c-b367-4263-966f-6e216aa97e26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:49.724291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0a3c-b367-4263-966f-6e216aa97e26 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:49.725072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0a3c-b367-4263-966f-6e216aa97e26 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:49.725480 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0a3c-b367-4263-966f-6e216aa97e26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:49.727640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:49.771089 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd0a3c-b367-4263-966f-6e216aa97e26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:49.782035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd0a3c-b367-4263-966f-6e216aa97e26 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:49.830232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb98a151879f44178106e60d99be0f10', 'request_id': 'req-050a3c70-40b9-43eb-a332-8052334339dc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:49.833945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 939/1878] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:49.845111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb98a151879f44178106e60d99be0f10', 'request_id': 'req-050a3c70-40b9-43eb-a332-8052334339dc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:49.863923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb98a151879f44178106e60d99be0f10', 'request_id': 'req-050a3c70-40b9-43eb-a332-8052334339dc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:49.880904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] The token cache is being invalidate because project eb98a151879f44178106e60d99be0f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:58:49.881099 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:49.882393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-050a3c70-40b9-43eb-a332-8052334339dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:49.886664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 940/1879] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:58:49 2022] DELETE /identity/v3/projects/eb98a151879f44178106e60d99be0f10 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:49.889107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527266e6-b973-4fa5-9827-fbb3d74103d5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:49.889456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527266e6-b973-4fa5-9827-fbb3d74103d5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:49.889801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527266e6-b973-4fa5-9827-fbb3d74103d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:49.914983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-527266e6-b973-4fa5-9827-fbb3d74103d5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:49.916351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-527266e6-b973-4fa5-9827-fbb3d74103d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:50.048262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 940/1880] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:50.065219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e8e57-9353-4d70-9bb8-84eba3a21d68 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:50.065569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e8e57-9353-4d70-9bb8-84eba3a21d68 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:50.065915 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e8e57-9353-4d70-9bb8-84eba3a21d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:50.095334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4e9e8e57-9353-4d70-9bb8-84eba3a21d68 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:50.096870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e9e8e57-9353-4d70-9bb8-84eba3a21d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:50.109502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 941/1881] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:50.113748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527266e6-b973-4fa5-9827-fbb3d74103d5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:50.134827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527266e6-b973-4fa5-9827-fbb3d74103d5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:50.144796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cbe23-4d08-4de5-9198-5b2570300a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:50.145535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cbe23-4d08-4de5-9198-5b2570300a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:50.147879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cbe23-4d08-4de5-9198-5b2570300a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:50.147879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cbe23-4d08-4de5-9198-5b2570300a08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:50.147879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489cbe23-4d08-4de5-9198-5b2570300a08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:58:50.147879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489cbe23-4d08-4de5-9198-5b2570300a08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:50.151498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489cbe23-4d08-4de5-9198-5b2570300a08 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:50.181693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 941/1882] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:58:50.187059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e8e57-9353-4d70-9bb8-84eba3a21d68 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:50.200047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e8e57-9353-4d70-9bb8-84eba3a21d68 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:50.210663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca365b2-b50b-408d-a32d-3db5c64993c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:50.212545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca365b2-b50b-408d-a32d-3db5c64993c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:50.213303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca365b2-b50b-408d-a32d-3db5c64993c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:50.214018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca365b2-b50b-408d-a32d-3db5c64993c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:50.214975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca365b2-b50b-408d-a32d-3db5c64993c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:50.218409 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca365b2-b50b-408d-a32d-3db5c64993c9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:50.291922 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 942/1883] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:58:50 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:58:50.296477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cbe23-4d08-4de5-9198-5b2570300a08 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:50.317083 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489cbe23-4d08-4de5-9198-5b2570300a08 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:50.378868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:50.425243 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4e955-b195-4a03-a86e-6dd5cafb134a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:50.425870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4e955-b195-4a03-a86e-6dd5cafb134a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:50.426154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4e955-b195-4a03-a86e-6dd5cafb134a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:50.426477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4e955-b195-4a03-a86e-6dd5cafb134a admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:50.427143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4e955-b195-4a03-a86e-6dd5cafb134a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:50.429835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4e955-b195-4a03-a86e-6dd5cafb134a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:50.461783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 942/1884] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:58:50 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:58:50.466527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca365b2-b50b-408d-a32d-3db5c64993c9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:50.490827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca365b2-b50b-408d-a32d-3db5c64993c9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:50.514184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3610-f41d-403f-9629-205b443e6319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:50.515024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3610-f41d-403f-9629-205b443e6319 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:50.515375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3610-f41d-403f-9629-205b443e6319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:50.515764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3610-f41d-403f-9629-205b443e6319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:50.516624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f3610-f41d-403f-9629-205b443e6319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:50.518552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f3610-f41d-403f-9629-205b443e6319 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:50.549250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 943/1885] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:50.552975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4e955-b195-4a03-a86e-6dd5cafb134a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:50.573344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4e955-b195-4a03-a86e-6dd5cafb134a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:50.595045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a42c37-69ad-43c2-b0f1-b1dbc006e686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:50.595819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a42c37-69ad-43c2-b0f1-b1dbc006e686 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:50.597305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a42c37-69ad-43c2-b0f1-b1dbc006e686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:50.597565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a42c37-69ad-43c2-b0f1-b1dbc006e686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:50.598285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a42c37-69ad-43c2-b0f1-b1dbc006e686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:50.601426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a42c37-69ad-43c2-b0f1-b1dbc006e686 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:50.617957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 943/1886] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:58:50.620296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3610-f41d-403f-9629-205b443e6319 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:50.633045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3610-f41d-403f-9629-205b443e6319 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:50.640062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e6f08-b9ee-4454-a0a0-399fabb0b338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:50.641169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e6f08-b9ee-4454-a0a0-399fabb0b338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:50.641924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e6f08-b9ee-4454-a0a0-399fabb0b338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:50.642496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e6f08-b9ee-4454-a0a0-399fabb0b338 admin admin] PATH_INFO: `/v3/projects/34670e3e0bb8474a84502e7710cbdc2c/users/a49b6f59d74240e78ea8fa180979c599/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:50.708284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e6f08-b9ee-4454-a0a0-399fabb0b338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34670e3e0bb8474a84502e7710cbdc2c, user_id=a49b6f59d74240e78ea8fa180979c599, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:50.714634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e6f08-b9ee-4454-a0a0-399fabb0b338 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:50.749266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 944/1887] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:58:50 2022] PUT /identity/v3/projects/34670e3e0bb8474a84502e7710cbdc2c/users/a49b6f59d74240e78ea8fa180979c599/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:58:50.760026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04fa80-e147-4b31-95ea-faa856b6ed78 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:50.760614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04fa80-e147-4b31-95ea-faa856b6ed78 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:50.761442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04fa80-e147-4b31-95ea-faa856b6ed78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:50.824027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7d04fa80-e147-4b31-95ea-faa856b6ed78 None None] MFA Rules not processed for user `a49b6f59d74240e78ea8fa180979c599`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:50.825212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7d04fa80-e147-4b31-95ea-faa856b6ed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:50.997967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 944/1888] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:51.014777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e6f08-b9ee-4454-a0a0-399fabb0b338 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:51.016840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-136e6f08-b9ee-4454-a0a0-399fabb0b338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:51.197246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e6f08-b9ee-4454-a0a0-399fabb0b338 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:51.202517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acfbe7-0c6e-4949-b170-23a252e304fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '548acf58bf9140ec9f1ba126e10deb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:51.202807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acfbe7-0c6e-4949-b170-23a252e304fc service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:51.203106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acfbe7-0c6e-4949-b170-23a252e304fc service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:51.203445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acfbe7-0c6e-4949-b170-23a252e304fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:51.216152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acfbe7-0c6e-4949-b170-23a252e304fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:51.217764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acfbe7-0c6e-4949-b170-23a252e304fc service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:51.229869 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 945/1889] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:58:59.693812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04fa80-e147-4b31-95ea-faa856b6ed78 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:59.695581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7d04fa80-e147-4b31-95ea-faa856b6ed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:59.762499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04fa80-e147-4b31-95ea-faa856b6ed78 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:58:59.766958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:58:59.767598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:59.767880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:59.768158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] PATH_INFO: `/v3/users/8394d05ebe404329ae33220c5d74d4e8` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:59.769329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8394d05ebe404329ae33220c5d74d4e8)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:58:59.771326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:58:59.808510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:59.823538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8394d05ebe404329ae33220c5d74d4e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:59.831712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8394d05ebe404329ae33220c5d74d4e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:59.832432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8394d05ebe404329ae33220c5d74d4e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:59.833048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8394d05ebe404329ae33220c5d74d4e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:59.845695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8394d05ebe404329ae33220c5d74d4e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:58:59.859968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 945/1890] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:58:59 2022] DELETE /identity/v3/users/8394d05ebe404329ae33220c5d74d4e8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:58:59.865130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c152115-0328-473d-b543-b478d475f153 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:58:59.865130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c152115-0328-473d-b543-b478d475f153 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:58:59.865130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c152115-0328-473d-b543-b478d475f153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:58:59.886539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0c152115-0328-473d-b543-b478d475f153 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:58:59.887793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c152115-0328-473d-b543-b478d475f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:58:59.898158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:58:59.963232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 946/1891] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:58:59.977191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:58:59.982529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:00.066730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9877b0-a18e-41ad-83e8-10b1c20accb8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:00.074299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3bc8b-28d5-4fdb-a783-4ba7d2593976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:00.075049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3bc8b-28d5-4fdb-a783-4ba7d2593976 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:00.075384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3bc8b-28d5-4fdb-a783-4ba7d2593976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:00.076257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3bc8b-28d5-4fdb-a783-4ba7d2593976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:00.094589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3bc8b-28d5-4fdb-a783-4ba7d2593976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:00.096059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3bc8b-28d5-4fdb-a783-4ba7d2593976 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:00.116645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 946/1892] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:00.504612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c152115-0328-473d-b543-b478d475f153 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:00.509174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c152115-0328-473d-b543-b478d475f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:00.544779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c152115-0328-473d-b543-b478d475f153 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:00.560780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:00.561622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:00.562071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:00.562662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] PATH_INFO: `/v3/projects/1c7feec501a9450ba2ed07ca24b01aa2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:00.565198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7feec501a9450ba2ed07ca24b01aa2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:00.571875 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:00.686833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7feec501a9450ba2ed07ca24b01aa2', 'request_id': 'req-8d541455-dbb3-4483-82bd-36cf87b6eb01'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:00.700243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7feec501a9450ba2ed07ca24b01aa2', 'request_id': 'req-8d541455-dbb3-4483-82bd-36cf87b6eb01'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:00.708320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7feec501a9450ba2ed07ca24b01aa2', 'request_id': 'req-8d541455-dbb3-4483-82bd-36cf87b6eb01'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:00.753629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] The token cache is being invalidate because project 1c7feec501a9450ba2ed07ca24b01aa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:00.753959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:00.755505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:00.759661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 947/1893] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:00 2022] DELETE /identity/v3/projects/1c7feec501a9450ba2ed07ca24b01aa2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:00.764013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3bc8b-28d5-4fdb-a783-4ba7d2593976 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:00.779196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3bc8b-28d5-4fdb-a783-4ba7d2593976 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:00.951242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:00.951901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:00.952263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:00.952456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] PATH_INFO: `/v3/users/22dc72956b344c0b833a34c96c8cbfe2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:00.955372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22dc72956b344c0b833a34c96c8cbfe2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:00.957323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:01.013470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22dc72956b344c0b833a34c96c8cbfe2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.018812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:01.021159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22dc72956b344c0b833a34c96c8cbfe2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.022545 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22dc72956b344c0b833a34c96c8cbfe2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.023779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22dc72956b344c0b833a34c96c8cbfe2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.037335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7f9384ca-a122-4980-8d32-07ea2e503e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22dc72956b344c0b833a34c96c8cbfe2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.051181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 947/1894] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:00 2022] DELETE /identity/v3/users/22dc72956b344c0b833a34c96c8cbfe2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:01.322768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:01.324258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:01.339925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d541455-dbb3-4483-82bd-36cf87b6eb01 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:01.413505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:01.414169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.414494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.414777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] PATH_INFO: `/v3/projects/da6396e6f3be4e829c4641baae5eaab8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:01.416038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da6396e6f3be4e829c4641baae5eaab8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:01.417977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:01.461158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:01.470017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da6396e6f3be4e829c4641baae5eaab8', 'request_id': 'req-b72472c2-3b40-4673-9caf-2689030848a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.478748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da6396e6f3be4e829c4641baae5eaab8', 'request_id': 'req-b72472c2-3b40-4673-9caf-2689030848a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.485172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da6396e6f3be4e829c4641baae5eaab8', 'request_id': 'req-b72472c2-3b40-4673-9caf-2689030848a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.497456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] The token cache is being invalidate because project da6396e6f3be4e829c4641baae5eaab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:01.497913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.498615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b72472c2-3b40-4673-9caf-2689030848a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:01.500354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 948/1895] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:01 2022] DELETE /identity/v3/projects/da6396e6f3be4e829c4641baae5eaab8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:01.505486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dba5a-c558-4707-b41a-1fccdac2f57f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.505869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dba5a-c558-4707-b41a-1fccdac2f57f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.506146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dba5a-c558-4707-b41a-1fccdac2f57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:01.529721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f76dba5a-c558-4707-b41a-1fccdac2f57f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:01.531238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f76dba5a-c558-4707-b41a-1fccdac2f57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:01.598366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 948/1896] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:01.607849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af992b17-afaa-414d-8de5-aca7eb7e166a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.608142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af992b17-afaa-414d-8de5-aca7eb7e166a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.608425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af992b17-afaa-414d-8de5-aca7eb7e166a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:01.628180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-af992b17-afaa-414d-8de5-aca7eb7e166a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:01.629385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-af992b17-afaa-414d-8de5-aca7eb7e166a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:01.637845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 949/1897] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:01.642663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dba5a-c558-4707-b41a-1fccdac2f57f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:01.654630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dba5a-c558-4707-b41a-1fccdac2f57f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:01.659531 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c4893-2d30-4de9-a0d8-9e38a8bb6e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:01.660156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c4893-2d30-4de9-a0d8-9e38a8bb6e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.660437 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c4893-2d30-4de9-a0d8-9e38a8bb6e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.660710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c4893-2d30-4de9-a0d8-9e38a8bb6e11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:01.661412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c4893-2d30-4de9-a0d8-9e38a8bb6e11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:01.661717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c4893-2d30-4de9-a0d8-9e38a8bb6e11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:01.664032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c4893-2d30-4de9-a0d8-9e38a8bb6e11 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:01.676244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 949/1898] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:59:01.680965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af992b17-afaa-414d-8de5-aca7eb7e166a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:01.691055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af992b17-afaa-414d-8de5-aca7eb7e166a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:01.697243 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886db75-befe-4bb7-a32b-1a6b90372337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:01.697888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886db75-befe-4bb7-a32b-1a6b90372337 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.698182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886db75-befe-4bb7-a32b-1a6b90372337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.698634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886db75-befe-4bb7-a32b-1a6b90372337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:01.699152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886db75-befe-4bb7-a32b-1a6b90372337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:01.701150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886db75-befe-4bb7-a32b-1a6b90372337 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:01.735633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 950/1899] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:01 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:01.740251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c4893-2d30-4de9-a0d8-9e38a8bb6e11 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:01.757099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c4893-2d30-4de9-a0d8-9e38a8bb6e11 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:01.823409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfa218-0e4e-45ea-b268-23afbc932ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:01.824044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfa218-0e4e-45ea-b268-23afbc932ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.824448 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfa218-0e4e-45ea-b268-23afbc932ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.824787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfa218-0e4e-45ea-b268-23afbc932ef7 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:01.825513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfa218-0e4e-45ea-b268-23afbc932ef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:01.827533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfa218-0e4e-45ea-b268-23afbc932ef7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:01.849688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 950/1900] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:01 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:01.854481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886db75-befe-4bb7-a32b-1a6b90372337 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:01.865065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886db75-befe-4bb7-a32b-1a6b90372337 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:01.870750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f365e-67d2-4ab5-a49f-bc3ab1cd7b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:01.871373 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f365e-67d2-4ab5-a49f-bc3ab1cd7b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.871651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f365e-67d2-4ab5-a49f-bc3ab1cd7b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.871920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f365e-67d2-4ab5-a49f-bc3ab1cd7b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:01.872616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f365e-67d2-4ab5-a49f-bc3ab1cd7b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:01.874206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f365e-67d2-4ab5-a49f-bc3ab1cd7b8e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:01.887178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 951/1901] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:01.888676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfa218-0e4e-45ea-b268-23afbc932ef7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:01.898782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfa218-0e4e-45ea-b268-23afbc932ef7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:01.903356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e174375-2ab0-4980-b7ba-06de1cae6bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:01.903954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e174375-2ab0-4980-b7ba-06de1cae6bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.904233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e174375-2ab0-4980-b7ba-06de1cae6bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.904500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e174375-2ab0-4980-b7ba-06de1cae6bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:01.905180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e174375-2ab0-4980-b7ba-06de1cae6bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:01.906810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e174375-2ab0-4980-b7ba-06de1cae6bb1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:01.917724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 951/1902] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:01.921998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f365e-67d2-4ab5-a49f-bc3ab1cd7b8e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:01.932668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f365e-67d2-4ab5-a49f-bc3ab1cd7b8e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:01.937485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2388f-6dd2-4992-a1ec-5c0275778a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:01.938156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2388f-6dd2-4992-a1ec-5c0275778a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.938452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2388f-6dd2-4992-a1ec-5c0275778a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.938737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2388f-6dd2-4992-a1ec-5c0275778a1b admin admin] PATH_INFO: `/v3/projects/885fbbaf24d24817b7592f7214ed54e6/users/80a71835993243149f4ebfbb422e261f/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:01.954374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2388f-6dd2-4992-a1ec-5c0275778a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=885fbbaf24d24817b7592f7214ed54e6, user_id=80a71835993243149f4ebfbb422e261f, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:01.957071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2388f-6dd2-4992-a1ec-5c0275778a1b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:01.971463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 952/1903] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:01 2022] PUT /identity/v3/projects/885fbbaf24d24817b7592f7214ed54e6/users/80a71835993243149f4ebfbb422e261f/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:01.982191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9d142-3e70-49c4-903a-7d43f7f6e14b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:01.982834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9d142-3e70-49c4-903a-7d43f7f6e14b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:01.983277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9d142-3e70-49c4-903a-7d43f7f6e14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:02.022453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-34b9d142-3e70-49c4-903a-7d43f7f6e14b None None] MFA Rules not processed for user `80a71835993243149f4ebfbb422e261f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:02.024107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-34b9d142-3e70-49c4-903a-7d43f7f6e14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:02.055557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:02.107237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 952/1904] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:02.119012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2388f-6dd2-4992-a1ec-5c0275778a1b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:02.120741 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-64a2388f-6dd2-4992-a1ec-5c0275778a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:02.182580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2388f-6dd2-4992-a1ec-5c0275778a1b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:02.186226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e68322-a863-43f9-8c94-3433f433e9e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:02.186892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e68322-a863-43f9-8c94-3433f433e9e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:02.187176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e68322-a863-43f9-8c94-3433f433e9e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:02.187454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e68322-a863-43f9-8c94-3433f433e9e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:02.196172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e68322-a863-43f9-8c94-3433f433e9e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:02.197644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e68322-a863-43f9-8c94-3433f433e9e3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:02.206636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 953/1905] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:02.226612 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9d142-3e70-49c4-903a-7d43f7f6e14b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:02.287314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9d142-3e70-49c4-903a-7d43f7f6e14b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:02.292633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:02.293269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:02.293551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:02.293827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] PATH_INFO: `/v3/users/80a71835993243149f4ebfbb422e261f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:02.295278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] RBAC: Authorizing `identity:delete_user(user_id=80a71835993243149f4ebfbb422e261f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:02.297200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:02.343198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80a71835993243149f4ebfbb422e261f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.350715 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80a71835993243149f4ebfbb422e261f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.351386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80a71835993243149f4ebfbb422e261f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.352016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80a71835993243149f4ebfbb422e261f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.363767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80a71835993243149f4ebfbb422e261f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.376338 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 953/1906] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:02 2022] DELETE /identity/v3/users/80a71835993243149f4ebfbb422e261f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:02.380997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78606f4-e127-4b2b-b5f4-6fa0874dd7af None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:02.381445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78606f4-e127-4b2b-b5f4-6fa0874dd7af None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:02.381594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78606f4-e127-4b2b-b5f4-6fa0874dd7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:02.401018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d78606f4-e127-4b2b-b5f4-6fa0874dd7af None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:02.402211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d78606f4-e127-4b2b-b5f4-6fa0874dd7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:02.464477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 954/1907] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:02.474073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:02.475529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:02.528176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:02.534540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323772fc-720e-434b-93b3-d191a148d24b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:02.539132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc279ad-229d-49d9-aaeb-75a4680a933a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:02.539784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc279ad-229d-49d9-aaeb-75a4680a933a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:02.540064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc279ad-229d-49d9-aaeb-75a4680a933a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:02.540339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc279ad-229d-49d9-aaeb-75a4680a933a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:02.549700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc279ad-229d-49d9-aaeb-75a4680a933a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:02.551168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc279ad-229d-49d9-aaeb-75a4680a933a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:02.560725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 954/1908] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:02.833211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78606f4-e127-4b2b-b5f4-6fa0874dd7af None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:02.835926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d78606f4-e127-4b2b-b5f4-6fa0874dd7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:02.852470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78606f4-e127-4b2b-b5f4-6fa0874dd7af None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:02.858069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:02.859636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:02.860007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:02.860679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] PATH_INFO: `/v3/projects/885fbbaf24d24817b7592f7214ed54e6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:02.861999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] RBAC: Authorizing `identity:delete_project(project_id=885fbbaf24d24817b7592f7214ed54e6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:02.864324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:02.867728 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:02.915590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885fbbaf24d24817b7592f7214ed54e6', 'request_id': 'req-be31e79d-9c6d-47a4-96ed-04b50256122f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.923691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885fbbaf24d24817b7592f7214ed54e6', 'request_id': 'req-be31e79d-9c6d-47a4-96ed-04b50256122f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.929742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885fbbaf24d24817b7592f7214ed54e6', 'request_id': 'req-be31e79d-9c6d-47a4-96ed-04b50256122f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.941390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] The token cache is being invalidate because project 885fbbaf24d24817b7592f7214ed54e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:02.941794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.942640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-be31e79d-9c6d-47a4-96ed-04b50256122f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:02.944640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 955/1909] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:02 2022] DELETE /identity/v3/projects/885fbbaf24d24817b7592f7214ed54e6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:02.949660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0b341-3681-4c3e-b5c8-6007d38efa18 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:02.950110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0b341-3681-4c3e-b5c8-6007d38efa18 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:02.950197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0b341-3681-4c3e-b5c8-6007d38efa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:02.970314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-96b0b341-3681-4c3e-b5c8-6007d38efa18 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:02.971561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-96b0b341-3681-4c3e-b5c8-6007d38efa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:03.044963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 955/1910] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:03.052611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d6c65-7436-4748-807a-2605a5e3d5ca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.052974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d6c65-7436-4748-807a-2605a5e3d5ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.054262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d6c65-7436-4748-807a-2605a5e3d5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.075043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8c5d6c65-7436-4748-807a-2605a5e3d5ca None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:03.076254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c5d6c65-7436-4748-807a-2605a5e3d5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:03.084063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 956/1911] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:03.088251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0b341-3681-4c3e-b5c8-6007d38efa18 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:03.098007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0b341-3681-4c3e-b5c8-6007d38efa18 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:03.103252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d90e-453a-4f73-a7f4-a714d6cb3788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:03.103877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d90e-453a-4f73-a7f4-a714d6cb3788 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.104162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d90e-453a-4f73-a7f4-a714d6cb3788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.104436 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d90e-453a-4f73-a7f4-a714d6cb3788 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.105123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648d90e-453a-4f73-a7f4-a714d6cb3788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:03.105423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648d90e-453a-4f73-a7f4-a714d6cb3788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:03.107051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648d90e-453a-4f73-a7f4-a714d6cb3788 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:03.120871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 956/1912] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:59:03.125293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d6c65-7436-4748-807a-2605a5e3d5ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:03.137446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d6c65-7436-4748-807a-2605a5e3d5ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:03.142303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49818431-9452-4ddc-8269-e043686eeb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:03.142974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49818431-9452-4ddc-8269-e043686eeb5d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.143258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49818431-9452-4ddc-8269-e043686eeb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.143527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49818431-9452-4ddc-8269-e043686eeb5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.144190 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49818431-9452-4ddc-8269-e043686eeb5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:03.146198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49818431-9452-4ddc-8269-e043686eeb5d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:03.175754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 957/1913] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:03 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:03.183662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d90e-453a-4f73-a7f4-a714d6cb3788 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:03.197009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d90e-453a-4f73-a7f4-a714d6cb3788 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:03.281082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e95a4-d281-4448-aefa-c605ec1ee03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:03.281712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e95a4-d281-4448-aefa-c605ec1ee03d admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.282048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e95a4-d281-4448-aefa-c605ec1ee03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.282331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e95a4-d281-4448-aefa-c605ec1ee03d admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.283029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e95a4-d281-4448-aefa-c605ec1ee03d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:03.285612 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e95a4-d281-4448-aefa-c605ec1ee03d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:03.324215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 957/1914] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:03 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:03.324316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49818431-9452-4ddc-8269-e043686eeb5d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:03.335451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49818431-9452-4ddc-8269-e043686eeb5d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:03.340200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20626061-6d50-4d13-b90d-d0cc33e8d390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:03.340816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20626061-6d50-4d13-b90d-d0cc33e8d390 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.341093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20626061-6d50-4d13-b90d-d0cc33e8d390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.341364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20626061-6d50-4d13-b90d-d0cc33e8d390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.342103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20626061-6d50-4d13-b90d-d0cc33e8d390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:03.343843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20626061-6d50-4d13-b90d-d0cc33e8d390 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:03.354192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 958/1915] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:03.357989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e95a4-d281-4448-aefa-c605ec1ee03d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:03.368275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e95a4-d281-4448-aefa-c605ec1ee03d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:03.373421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02937b-6b63-4e39-b487-c28baa4c4853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:03.374030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02937b-6b63-4e39-b487-c28baa4c4853 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.374311 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02937b-6b63-4e39-b487-c28baa4c4853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.374673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02937b-6b63-4e39-b487-c28baa4c4853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.375421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02937b-6b63-4e39-b487-c28baa4c4853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:03.377129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02937b-6b63-4e39-b487-c28baa4c4853 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:03.389194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 958/1916] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:03.393737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20626061-6d50-4d13-b90d-d0cc33e8d390 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:03.406076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20626061-6d50-4d13-b90d-d0cc33e8d390 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:03.410729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61beae80-6069-46fe-8a5f-580d6ea6b6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:03.411389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61beae80-6069-46fe-8a5f-580d6ea6b6bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.411667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61beae80-6069-46fe-8a5f-580d6ea6b6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.411939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61beae80-6069-46fe-8a5f-580d6ea6b6bb admin admin] PATH_INFO: `/v3/projects/63bf9accf0964034a393028eb974bf58/users/56d68a23151a4ab2bbcf13de4028e2c4/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.426663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61beae80-6069-46fe-8a5f-580d6ea6b6bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=63bf9accf0964034a393028eb974bf58, user_id=56d68a23151a4ab2bbcf13de4028e2c4, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:03.429611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61beae80-6069-46fe-8a5f-580d6ea6b6bb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:03.442387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 959/1917] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:03 2022] PUT /identity/v3/projects/63bf9accf0964034a393028eb974bf58/users/56d68a23151a4ab2bbcf13de4028e2c4/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:03.454127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f914a41-5ccd-4600-9a02-1128f35aaa83 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.454304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f914a41-5ccd-4600-9a02-1128f35aaa83 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.454624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f914a41-5ccd-4600-9a02-1128f35aaa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.482335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3f914a41-5ccd-4600-9a02-1128f35aaa83 None None] MFA Rules not processed for user `56d68a23151a4ab2bbcf13de4028e2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:03.483586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3f914a41-5ccd-4600-9a02-1128f35aaa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:03.527193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:03.552067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 959/1918] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:03.563906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61beae80-6069-46fe-8a5f-580d6ea6b6bb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:03.565629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-61beae80-6069-46fe-8a5f-580d6ea6b6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:03.622934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61beae80-6069-46fe-8a5f-580d6ea6b6bb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:03.626729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce45c9d-8fb1-4d1d-a2cb-76b01113cb6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:03.627290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce45c9d-8fb1-4d1d-a2cb-76b01113cb6a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.627571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce45c9d-8fb1-4d1d-a2cb-76b01113cb6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.627846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce45c9d-8fb1-4d1d-a2cb-76b01113cb6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.637231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce45c9d-8fb1-4d1d-a2cb-76b01113cb6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:03.638682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce45c9d-8fb1-4d1d-a2cb-76b01113cb6a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:03.648824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 960/1919] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:03.675173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f914a41-5ccd-4600-9a02-1128f35aaa83 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:03.736313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f914a41-5ccd-4600-9a02-1128f35aaa83 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:03.741786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:03.742423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.742722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.742989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] PATH_INFO: `/v3/users/56d68a23151a4ab2bbcf13de4028e2c4` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.744160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d68a23151a4ab2bbcf13de4028e2c4)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:03.746066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:03.798882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d68a23151a4ab2bbcf13de4028e2c4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:03.806806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d68a23151a4ab2bbcf13de4028e2c4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:03.807656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d68a23151a4ab2bbcf13de4028e2c4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:03.808371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d68a23151a4ab2bbcf13de4028e2c4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:03.821780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d68a23151a4ab2bbcf13de4028e2c4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:03.836038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 960/1920] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:03 2022] DELETE /identity/v3/users/56d68a23151a4ab2bbcf13de4028e2c4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:03.840662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0814d-5f69-4a90-aaf7-1533ad976e04 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:03.841071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0814d-5f69-4a90-aaf7-1533ad976e04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:03.841540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0814d-5f69-4a90-aaf7-1533ad976e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:03.861238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-33a0814d-5f69-4a90-aaf7-1533ad976e04 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:03.862484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-33a0814d-5f69-4a90-aaf7-1533ad976e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:03.899167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:03.938806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 961/1921] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:59:03.948986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:03.950430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:04.009798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895883c-330f-486a-aa06-06f0eb3a91ff admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:04.013427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8a94e-0b50-4a93-824f-c1202f354ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:04.014047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8a94e-0b50-4a93-824f-c1202f354ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.014325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8a94e-0b50-4a93-824f-c1202f354ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.014619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8a94e-0b50-4a93-824f-c1202f354ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.023939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8a94e-0b50-4a93-824f-c1202f354ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:04.025381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8a94e-0b50-4a93-824f-c1202f354ee9 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:04.034975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 961/1922] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:04.338134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b97bb-59b9-4ecf-97c6-dfb58955498c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.338469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b97bb-59b9-4ecf-97c6-dfb58955498c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.341339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b97bb-59b9-4ecf-97c6-dfb58955498c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.341339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 962/1923] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 04:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 04:59:04.354079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de28f1-ca71-4ad2-b81d-59e5ff5ad008 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.354079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de28f1-ca71-4ad2-b81d-59e5ff5ad008 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.354630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de28f1-ca71-4ad2-b81d-59e5ff5ad008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.375025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:04.383361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c3de28f1-ca71-4ad2-b81d-59e5ff5ad008 None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:04.384726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c3de28f1-ca71-4ad2-b81d-59e5ff5ad008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:04.408543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b97bb-59b9-4ecf-97c6-dfb58955498c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:04.410591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-983b97bb-59b9-4ecf-97c6-dfb58955498c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:04.447108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b97bb-59b9-4ecf-97c6-dfb58955498c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:04.455531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:04.456474 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.456901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.457392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] PATH_INFO: `/v3/projects/63bf9accf0964034a393028eb974bf58` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.460614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63bf9accf0964034a393028eb974bf58)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:04.463199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:04.531516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 962/1924] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 04:59:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:04.587030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de28f1-ca71-4ad2-b81d-59e5ff5ad008 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:04.592971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63bf9accf0964034a393028eb974bf58', 'request_id': 'req-8a0f2f2b-a298-489c-8238-fa1e2b874d11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:04.593455 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c3de28f1-ca71-4ad2-b81d-59e5ff5ad008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:04.603136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63bf9accf0964034a393028eb974bf58', 'request_id': 'req-8a0f2f2b-a298-489c-8238-fa1e2b874d11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:04.610239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de28f1-ca71-4ad2-b81d-59e5ff5ad008 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:04.610396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63bf9accf0964034a393028eb974bf58', 'request_id': 'req-8a0f2f2b-a298-489c-8238-fa1e2b874d11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:04.627246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] The token cache is being invalidate because project 63bf9accf0964034a393028eb974bf58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:04.627780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:04.628443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a0f2f2b-a298-489c-8238-fa1e2b874d11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:04.629986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 963/1925] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:04 2022] DELETE /identity/v3/projects/63bf9accf0964034a393028eb974bf58 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:04.634793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc0921-5b8f-41a1-a4bb-9093de52f55a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.635139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc0921-5b8f-41a1-a4bb-9093de52f55a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.635416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc0921-5b8f-41a1-a4bb-9093de52f55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.655476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d4cc0921-5b8f-41a1-a4bb-9093de52f55a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:04.656611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4cc0921-5b8f-41a1-a4bb-9093de52f55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:04.683145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f899dc-8589-454c-bb36-e85eafc314b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:04.683768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f899dc-8589-454c-bb36-e85eafc314b4 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.684045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f899dc-8589-454c-bb36-e85eafc314b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.684316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f899dc-8589-454c-bb36-e85eafc314b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.686090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b6f899dc-8589-454c-bb36-e85eafc314b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:04.698491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f899dc-8589-454c-bb36-e85eafc314b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:04.700070 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f899dc-8589-454c-bb36-e85eafc314b4 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:04.711186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 963/1926] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:04.717780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 964/1927] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:04.726397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63f7d8-bee8-48ec-a131-15e49ec450a9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.726753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63f7d8-bee8-48ec-a131-15e49ec450a9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.727030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63f7d8-bee8-48ec-a131-15e49ec450a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.747400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-df63f7d8-bee8-48ec-a131-15e49ec450a9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:04.748625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-df63f7d8-bee8-48ec-a131-15e49ec450a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:04.756723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 964/1928] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:04.760946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc0921-5b8f-41a1-a4bb-9093de52f55a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:04.770795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc0921-5b8f-41a1-a4bb-9093de52f55a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:04.775439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9a2a9-043e-4519-90ed-36dc063e843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:04.776109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9a2a9-043e-4519-90ed-36dc063e843c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.776342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9a2a9-043e-4519-90ed-36dc063e843c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.776638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9a2a9-043e-4519-90ed-36dc063e843c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.777452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9a2a9-043e-4519-90ed-36dc063e843c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:04.777718 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9a2a9-043e-4519-90ed-36dc063e843c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:04.779456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9a2a9-043e-4519-90ed-36dc063e843c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:04.791707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 965/1929] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:59:04.796280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63f7d8-bee8-48ec-a131-15e49ec450a9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:04.806511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63f7d8-bee8-48ec-a131-15e49ec450a9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:04.811308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced6023-d1da-4fc6-a16a-d71d2d074cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:04.811912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced6023-d1da-4fc6-a16a-d71d2d074cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.812192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced6023-d1da-4fc6-a16a-d71d2d074cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.812466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced6023-d1da-4fc6-a16a-d71d2d074cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.813130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced6023-d1da-4fc6-a16a-d71d2d074cb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:04.815118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced6023-d1da-4fc6-a16a-d71d2d074cb8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:04.843083 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 965/1930] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:04 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:04.847160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9a2a9-043e-4519-90ed-36dc063e843c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:04.857609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9a2a9-043e-4519-90ed-36dc063e843c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:04.897689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:04.915498 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9ae0e-3d17-4ec5-8438-c11b008c1df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:04.916048 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9ae0e-3d17-4ec5-8438-c11b008c1df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.916325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9ae0e-3d17-4ec5-8438-c11b008c1df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.916592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9ae0e-3d17-4ec5-8438-c11b008c1df9 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.917254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9ae0e-3d17-4ec5-8438-c11b008c1df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:04.919218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9ae0e-3d17-4ec5-8438-c11b008c1df9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:04.942057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 966/1931] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:04 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:04.946586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced6023-d1da-4fc6-a16a-d71d2d074cb8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:04.958236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced6023-d1da-4fc6-a16a-d71d2d074cb8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:04.964126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee87cf9-2521-449f-8c33-7ca5450a857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:04.964834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee87cf9-2521-449f-8c33-7ca5450a857c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.965182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee87cf9-2521-449f-8c33-7ca5450a857c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.965530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee87cf9-2521-449f-8c33-7ca5450a857c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:04.966417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee87cf9-2521-449f-8c33-7ca5450a857c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:04.968234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee87cf9-2521-449f-8c33-7ca5450a857c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:04.977011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9ae0e-3d17-4ec5-8438-c11b008c1df9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:04.978849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ea9ae0e-3d17-4ec5-8438-c11b008c1df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:04.979843 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 966/1932] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:04.983624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee87cf9-2521-449f-8c33-7ca5450a857c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:04.993840 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee87cf9-2521-449f-8c33-7ca5450a857c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:04.995019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9ae0e-3d17-4ec5-8438-c11b008c1df9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:04.998699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcdacc-e297-4398-89a5-7761411817c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:04.999384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcdacc-e297-4398-89a5-7761411817c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:04.999736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcdacc-e297-4398-89a5-7761411817c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:04.999999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:05.000134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcdacc-e297-4398-89a5-7761411817c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:05.000770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:05.000963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dcdacc-e297-4398-89a5-7761411817c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:05.001185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:05.001531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] PATH_INFO: `/v3/users/bc4be97e0f274b54a29ba6d9df302ede` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:05.002821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dcdacc-e297-4398-89a5-7761411817c5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:05.003662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc4be97e0f274b54a29ba6d9df302ede)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:05.005791 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:05.015009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 967/1933] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:05.019613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcdacc-e297-4398-89a5-7761411817c5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:05.032796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcdacc-e297-4398-89a5-7761411817c5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:05.038605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2f24d-b5f8-4d5a-b9b5-48ad5cae9417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:05.039417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f24d-b5f8-4d5a-b9b5-48ad5cae9417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:05.039700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f24d-b5f8-4d5a-b9b5-48ad5cae9417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:05.039978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2f24d-b5f8-4d5a-b9b5-48ad5cae9417 admin admin] PATH_INFO: `/v3/projects/4d3dd8c4c5e84003801865b3cb7afd5c/users/425ecb85e0ab46888481118ba400edba/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:05.058687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2f24d-b5f8-4d5a-b9b5-48ad5cae9417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3dd8c4c5e84003801865b3cb7afd5c, user_id=425ecb85e0ab46888481118ba400edba, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:05.061830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2f24d-b5f8-4d5a-b9b5-48ad5cae9417 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:05.069436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc4be97e0f274b54a29ba6d9df302ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.078733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc4be97e0f274b54a29ba6d9df302ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.080008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc4be97e0f274b54a29ba6d9df302ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.080641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc4be97e0f274b54a29ba6d9df302ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.092496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 968/1934] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:05 2022] PUT /identity/v3/projects/4d3dd8c4c5e84003801865b3cb7afd5c/users/425ecb85e0ab46888481118ba400edba/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:05.095098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c74fc0a3-b0a0-435f-9373-e8edce9099b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc4be97e0f274b54a29ba6d9df302ede'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.103126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd47fc3-82f6-426b-8179-7a65d19869ff None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:05.103447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd47fc3-82f6-426b-8179-7a65d19869ff None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:05.103940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd47fc3-82f6-426b-8179-7a65d19869ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:05.116909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 967/1935] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:04 2022] DELETE /identity/v3/users/bc4be97e0f274b54a29ba6d9df302ede => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:05.120991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b72c2-3b1b-435f-8d46-967999643574 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:05.121278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b72c2-3b1b-435f-8d46-967999643574 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:05.121553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b72c2-3b1b-435f-8d46-967999643574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:05.133629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:05.141259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-428b72c2-3b1b-435f-8d46-967999643574 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:05.142511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-428b72c2-3b1b-435f-8d46-967999643574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:05.144523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fcd47fc3-82f6-426b-8179-7a65d19869ff None None] MFA Rules not processed for user `425ecb85e0ab46888481118ba400edba`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:05.145927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fcd47fc3-82f6-426b-8179-7a65d19869ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:05.203801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 968/1936] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:59:05.211775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 969/1937] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:05.215743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b72c2-3b1b-435f-8d46-967999643574 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:05.217202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-428b72c2-3b1b-435f-8d46-967999643574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:05.222152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd47fc3-82f6-426b-8179-7a65d19869ff None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:05.224049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fcd47fc3-82f6-426b-8179-7a65d19869ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:05.264881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:05.278617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b72c2-3b1b-435f-8d46-967999643574 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:05.282876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd47fc3-82f6-426b-8179-7a65d19869ff None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:05.283296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1972a4-3a52-42bf-91ce-441c683e3851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:05.283920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1972a4-3a52-42bf-91ce-441c683e3851 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:05.284205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1972a4-3a52-42bf-91ce-441c683e3851 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:05.284481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1972a4-3a52-42bf-91ce-441c683e3851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:05.286616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66c657-674c-4daf-9a92-e54a16166594 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:05.287289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66c657-674c-4daf-9a92-e54a16166594 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:05.287579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66c657-674c-4daf-9a92-e54a16166594 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:05.287852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66c657-674c-4daf-9a92-e54a16166594 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:05.295030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1972a4-3a52-42bf-91ce-441c683e3851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:05.296479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1972a4-3a52-42bf-91ce-441c683e3851 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:05.296611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66c657-674c-4daf-9a92-e54a16166594 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:05.297991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66c657-674c-4daf-9a92-e54a16166594 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:05.307032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 969/1938] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:59:05.307821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 970/1939] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:59:05.736707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1972a4-3a52-42bf-91ce-441c683e3851 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:05.738178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b1972a4-3a52-42bf-91ce-441c683e3851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:05.753306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1972a4-3a52-42bf-91ce-441c683e3851 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:05.761327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:05.762183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:05.762562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:05.762912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] PATH_INFO: `/v3/projects/897900152f9e4d8090677c6da2474a32` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:05.765250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=897900152f9e4d8090677c6da2474a32)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:05.768640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:05.839181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897900152f9e4d8090677c6da2474a32', 'request_id': 'req-3a2099bb-ab95-4927-9747-53c008545654'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.848771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897900152f9e4d8090677c6da2474a32', 'request_id': 'req-3a2099bb-ab95-4927-9747-53c008545654'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.855817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897900152f9e4d8090677c6da2474a32', 'request_id': 'req-3a2099bb-ab95-4927-9747-53c008545654'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.866966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] The token cache is being invalidate because project 897900152f9e4d8090677c6da2474a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:05.867253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.867880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.869378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 970/1940] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:05 2022] DELETE /identity/v3/projects/897900152f9e4d8090677c6da2474a32 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:05.873228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66c657-674c-4daf-9a92-e54a16166594 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:05.883914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66c657-674c-4daf-9a92-e54a16166594 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:05.939972 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:05.940615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:05.940897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:05.941173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] PATH_INFO: `/v3/users/7a5cb8c2a37042658c78342b612968fc` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:05.942349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5cb8c2a37042658c78342b612968fc)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:05.944340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:05.988192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:05.991611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5cb8c2a37042658c78342b612968fc'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.999189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5cb8c2a37042658c78342b612968fc'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:05.999850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5cb8c2a37042658c78342b612968fc'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:06.000498 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5cb8c2a37042658c78342b612968fc'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:06.013102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-545307e4-03b2-4d7b-bf55-0c3a441e633e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5cb8c2a37042658c78342b612968fc'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:06.027534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 971/1941] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:05 2022] DELETE /identity/v3/users/7a5cb8c2a37042658c78342b612968fc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:06.339618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:06.341821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:06.356785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2099bb-ab95-4927-9747-53c008545654 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:06.419974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:06.420627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:06.420905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:06.421172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] PATH_INFO: `/v3/projects/def3ff57a65d4cc49640e727f34498f2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:06.422331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=def3ff57a65d4cc49640e727f34498f2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:06.424260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:06.476517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def3ff57a65d4cc49640e727f34498f2', 'request_id': 'req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:06.485072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def3ff57a65d4cc49640e727f34498f2', 'request_id': 'req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:06.491228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def3ff57a65d4cc49640e727f34498f2', 'request_id': 'req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:06.498897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:06.503386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] The token cache is being invalidate because project def3ff57a65d4cc49640e727f34498f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:06.504033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:06.504978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffc4baef-fc7b-49e7-b6cf-8115535b2db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:06.507747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 971/1942] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:06 2022] DELETE /identity/v3/projects/def3ff57a65d4cc49640e727f34498f2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:06.512323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ee463-c1a8-4247-89f8-49830cd4efb2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:06.512609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ee463-c1a8-4247-89f8-49830cd4efb2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:06.512883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ee463-c1a8-4247-89f8-49830cd4efb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:06.533036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dc1ee463-c1a8-4247-89f8-49830cd4efb2 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:06.534235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dc1ee463-c1a8-4247-89f8-49830cd4efb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:06.600507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 972/1943] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:06.608328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d16fd1-5294-477a-b970-b7c648f8c23c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:06.608627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d16fd1-5294-477a-b970-b7c648f8c23c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:06.608909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d16fd1-5294-477a-b970-b7c648f8c23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:06.630468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-22d16fd1-5294-477a-b970-b7c648f8c23c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:06.632349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-22d16fd1-5294-477a-b970-b7c648f8c23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:06.641268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 972/1944] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:59:06.646069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ee463-c1a8-4247-89f8-49830cd4efb2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:06.656407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ee463-c1a8-4247-89f8-49830cd4efb2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:06.663113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff9cf7-1681-4cf5-9a33-8d1f48bf097e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:06.663736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff9cf7-1681-4cf5-9a33-8d1f48bf097e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:06.664093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff9cf7-1681-4cf5-9a33-8d1f48bf097e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:06.664433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff9cf7-1681-4cf5-9a33-8d1f48bf097e admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:06.665446 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff9cf7-1681-4cf5-9a33-8d1f48bf097e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:06.665849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff9cf7-1681-4cf5-9a33-8d1f48bf097e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:06.668127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff9cf7-1681-4cf5-9a33-8d1f48bf097e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:06.688929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 973/1945] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:59:06.693284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d16fd1-5294-477a-b970-b7c648f8c23c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:06.714823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d16fd1-5294-477a-b970-b7c648f8c23c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:06.742734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600d44f-a09e-4fb6-8b23-d3cd629481df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:06.742734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600d44f-a09e-4fb6-8b23-d3cd629481df admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:06.742734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600d44f-a09e-4fb6-8b23-d3cd629481df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:06.742734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600d44f-a09e-4fb6-8b23-d3cd629481df admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:06.743459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600d44f-a09e-4fb6-8b23-d3cd629481df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:06.743459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600d44f-a09e-4fb6-8b23-d3cd629481df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:06.783148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 973/1946] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:06 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:06.787478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff9cf7-1681-4cf5-9a33-8d1f48bf097e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:06.798468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff9cf7-1681-4cf5-9a33-8d1f48bf097e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:06.865014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c15441-99ca-458c-b0a4-d10b59841059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:06.866500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c15441-99ca-458c-b0a4-d10b59841059 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:06.866794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c15441-99ca-458c-b0a4-d10b59841059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:06.867037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c15441-99ca-458c-b0a4-d10b59841059 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:06.867726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c15441-99ca-458c-b0a4-d10b59841059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:06.869768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c15441-99ca-458c-b0a4-d10b59841059 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:06.899114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 974/1947] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:06 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:06.906785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600d44f-a09e-4fb6-8b23-d3cd629481df admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:06.918477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600d44f-a09e-4fb6-8b23-d3cd629481df admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:06.923608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0171682-7d90-4420-a2b5-e8fe354738b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:06.924239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0171682-7d90-4420-a2b5-e8fe354738b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:06.924521 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0171682-7d90-4420-a2b5-e8fe354738b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:06.924795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0171682-7d90-4420-a2b5-e8fe354738b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:06.925519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0171682-7d90-4420-a2b5-e8fe354738b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:06.927168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0171682-7d90-4420-a2b5-e8fe354738b9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:06.938554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 974/1948] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:06.942343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c15441-99ca-458c-b0a4-d10b59841059 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:06.953752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c15441-99ca-458c-b0a4-d10b59841059 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:06.959039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df5c8a-bb76-4634-a9a8-23659e6aacd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:06.960337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df5c8a-bb76-4634-a9a8-23659e6aacd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:06.960714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df5c8a-bb76-4634-a9a8-23659e6aacd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:06.961049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df5c8a-bb76-4634-a9a8-23659e6aacd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:06.961818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df5c8a-bb76-4634-a9a8-23659e6aacd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:06.963523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df5c8a-bb76-4634-a9a8-23659e6aacd5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:06.977017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 975/1949] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:06.981151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0171682-7d90-4420-a2b5-e8fe354738b9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:06.993688 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0171682-7d90-4420-a2b5-e8fe354738b9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:06.998984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae504ae6-a2f7-46c8-b20b-42977b37323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:07.000119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae504ae6-a2f7-46c8-b20b-42977b37323d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.000524 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae504ae6-a2f7-46c8-b20b-42977b37323d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.000801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae504ae6-a2f7-46c8-b20b-42977b37323d admin admin] PATH_INFO: `/v3/projects/8307d1ce91cf4313b41be212733981a5/users/73773e48d2444895b0abacaaebba60c7/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.024478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae504ae6-a2f7-46c8-b20b-42977b37323d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8307d1ce91cf4313b41be212733981a5, user_id=73773e48d2444895b0abacaaebba60c7, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:07.027306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae504ae6-a2f7-46c8-b20b-42977b37323d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:07.043428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 975/1950] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:06 2022] PUT /identity/v3/projects/8307d1ce91cf4313b41be212733981a5/users/73773e48d2444895b0abacaaebba60c7/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:07.055684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa853455-32f6-4402-8d7d-8ccb64ffe9b1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.056174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa853455-32f6-4402-8d7d-8ccb64ffe9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.056258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa853455-32f6-4402-8d7d-8ccb64ffe9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.085554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fa853455-32f6-4402-8d7d-8ccb64ffe9b1 None None] MFA Rules not processed for user `73773e48d2444895b0abacaaebba60c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:07.086979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fa853455-32f6-4402-8d7d-8ccb64ffe9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:07.100785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:07.154759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 976/1951] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:07.159379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae504ae6-a2f7-46c8-b20b-42977b37323d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:07.222354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae504ae6-a2f7-46c8-b20b-42977b37323d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:07.227602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c092-1058-4bd6-9156-f0471cc9705d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:07.228213 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c092-1058-4bd6-9156-f0471cc9705d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.228492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c092-1058-4bd6-9156-f0471cc9705d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.228765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c092-1058-4bd6-9156-f0471cc9705d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.229459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083c092-1058-4bd6-9156-f0471cc9705d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:07.231393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083c092-1058-4bd6-9156-f0471cc9705d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:07.260054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 976/1952] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:07 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:07.264058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa853455-32f6-4402-8d7d-8ccb64ffe9b1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:07.278549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa853455-32f6-4402-8d7d-8ccb64ffe9b1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:07.381834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674288f2-a35e-4dd1-9920-2cd57914f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:07.382486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674288f2-a35e-4dd1-9920-2cd57914f605 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.382763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674288f2-a35e-4dd1-9920-2cd57914f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.383029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674288f2-a35e-4dd1-9920-2cd57914f605 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.383771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674288f2-a35e-4dd1-9920-2cd57914f605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:07.386210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674288f2-a35e-4dd1-9920-2cd57914f605 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:07.409583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 977/1953] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:07 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:07.414602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c092-1058-4bd6-9156-f0471cc9705d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:07.427330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c092-1058-4bd6-9156-f0471cc9705d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:07.432998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bd7e7-375c-43c6-aba9-6a25d0106326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:07.433629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bd7e7-375c-43c6-aba9-6a25d0106326 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.433911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bd7e7-375c-43c6-aba9-6a25d0106326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.434184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bd7e7-375c-43c6-aba9-6a25d0106326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.434988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bd7e7-375c-43c6-aba9-6a25d0106326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:07.436589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bd7e7-375c-43c6-aba9-6a25d0106326 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:07.449062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 977/1954] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:07.453154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674288f2-a35e-4dd1-9920-2cd57914f605 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:07.464710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674288f2-a35e-4dd1-9920-2cd57914f605 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:07.469917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280f45d-b759-449d-9992-e3ae38800183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:07.470642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280f45d-b759-449d-9992-e3ae38800183 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.470987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280f45d-b759-449d-9992-e3ae38800183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.471329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280f45d-b759-449d-9992-e3ae38800183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.472094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280f45d-b759-449d-9992-e3ae38800183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:07.473763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280f45d-b759-449d-9992-e3ae38800183 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:07.485525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 978/1955] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:07.489724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bd7e7-375c-43c6-aba9-6a25d0106326 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:07.500188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bd7e7-375c-43c6-aba9-6a25d0106326 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:07.504883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48699723-571a-412b-909e-8ee42fb03127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:07.505543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699723-571a-412b-909e-8ee42fb03127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.505865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699723-571a-412b-909e-8ee42fb03127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.506135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699723-571a-412b-909e-8ee42fb03127 admin admin] PATH_INFO: `/v3/projects/7cc1724a09884b3887fd83e1eba1fe1f/users/9c45ba921c5f4cb38bda0a81ab11e425/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.522315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48699723-571a-412b-909e-8ee42fb03127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc1724a09884b3887fd83e1eba1fe1f, user_id=9c45ba921c5f4cb38bda0a81ab11e425, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:07.525077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48699723-571a-412b-909e-8ee42fb03127 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:07.542674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 978/1956] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:07 2022] PUT /identity/v3/projects/7cc1724a09884b3887fd83e1eba1fe1f/users/9c45ba921c5f4cb38bda0a81ab11e425/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:07.554563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d4975-ea2c-41f6-b5e0-86d6ee1721da None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.554851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d4975-ea2c-41f6-b5e0-86d6ee1721da None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.555126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d4975-ea2c-41f6-b5e0-86d6ee1721da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.582491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-580d4975-ea2c-41f6-b5e0-86d6ee1721da None None] MFA Rules not processed for user `9c45ba921c5f4cb38bda0a81ab11e425`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:07.583708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-580d4975-ea2c-41f6-b5e0-86d6ee1721da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:07.653147 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 979/1957] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:07.657605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48699723-571a-412b-909e-8ee42fb03127 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:07.711254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:07.727308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48699723-571a-412b-909e-8ee42fb03127 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:07.732278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512c878-d3b3-43a7-b8d8-8baa05d5c0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:07.732998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512c878-d3b3-43a7-b8d8-8baa05d5c0af admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.733576 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512c878-d3b3-43a7-b8d8-8baa05d5c0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.733869 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512c878-d3b3-43a7-b8d8-8baa05d5c0af admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.734841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512c878-d3b3-43a7-b8d8-8baa05d5c0af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:07.736950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512c878-d3b3-43a7-b8d8-8baa05d5c0af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:07.773923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 979/1958] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:07 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:07.778857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d4975-ea2c-41f6-b5e0-86d6ee1721da None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:07.792920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d4975-ea2c-41f6-b5e0-86d6ee1721da None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:07.825406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:07.914418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebb97d-aa72-4771-b778-05b3140edbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:07.915302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebb97d-aa72-4771-b778-05b3140edbbd admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.915652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebb97d-aa72-4771-b778-05b3140edbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.915990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebb97d-aa72-4771-b778-05b3140edbbd admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.916739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebb97d-aa72-4771-b778-05b3140edbbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:07.919990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebb97d-aa72-4771-b778-05b3140edbbd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:07.953745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 980/1959] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:07 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:07.958152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512c878-d3b3-43a7-b8d8-8baa05d5c0af admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:07.973026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512c878-d3b3-43a7-b8d8-8baa05d5c0af admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:07.980027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138a078-9a8c-49fa-ad3b-054df9e668f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:07.981043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138a078-9a8c-49fa-ad3b-054df9e668f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:07.981675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138a078-9a8c-49fa-ad3b-054df9e668f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:07.982277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138a078-9a8c-49fa-ad3b-054df9e668f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:07.983363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138a078-9a8c-49fa-ad3b-054df9e668f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:07.985580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138a078-9a8c-49fa-ad3b-054df9e668f2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:08.001726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 980/1960] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:08.007424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebb97d-aa72-4771-b778-05b3140edbbd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:08.022288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebb97d-aa72-4771-b778-05b3140edbbd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:08.027727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81815155-1394-4220-a82d-8794b2af7d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:08.028514 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81815155-1394-4220-a82d-8794b2af7d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:08.028714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81815155-1394-4220-a82d-8794b2af7d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:08.029080 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81815155-1394-4220-a82d-8794b2af7d2e admin admin] PATH_INFO: `/v3/projects/5df47d81e9ab4196abf4832a46739298/users/c80666a362864deab2e19c2d795311e8/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:08.050383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81815155-1394-4220-a82d-8794b2af7d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df47d81e9ab4196abf4832a46739298, user_id=c80666a362864deab2e19c2d795311e8, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:08.055274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81815155-1394-4220-a82d-8794b2af7d2e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:08.090134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 981/1961] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:08 2022] PUT /identity/v3/projects/5df47d81e9ab4196abf4832a46739298/users/c80666a362864deab2e19c2d795311e8/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:08.094266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138a078-9a8c-49fa-ad3b-054df9e668f2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:08.119567 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138a078-9a8c-49fa-ad3b-054df9e668f2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:08.290777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2fbe6-ba91-48a2-bbac-2c6dc53a142f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:08.291759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2fbe6-ba91-48a2-bbac-2c6dc53a142f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:08.292172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2fbe6-ba91-48a2-bbac-2c6dc53a142f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:08.292715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2fbe6-ba91-48a2-bbac-2c6dc53a142f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:08.294216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2fbe6-ba91-48a2-bbac-2c6dc53a142f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:08.299649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2fbe6-ba91-48a2-bbac-2c6dc53a142f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:08.312553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 981/1962] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:08.316170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81815155-1394-4220-a82d-8794b2af7d2e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:08.337584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81815155-1394-4220-a82d-8794b2af7d2e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:08.351367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24054e24-b0a0-43c0-beff-c3df64d2007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:08.355557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24054e24-b0a0-43c0-beff-c3df64d2007c admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:08.355933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24054e24-b0a0-43c0-beff-c3df64d2007c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:08.356419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24054e24-b0a0-43c0-beff-c3df64d2007c admin admin] PATH_INFO: `/v3/projects/5df47d81e9ab4196abf4832a46739298/users/c80666a362864deab2e19c2d795311e8/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:08.361807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24054e24-b0a0-43c0-beff-c3df64d2007c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df47d81e9ab4196abf4832a46739298, user_id=c80666a362864deab2e19c2d795311e8, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:08.369653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24054e24-b0a0-43c0-beff-c3df64d2007c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:08.414585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 982/1963] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:08 2022] PUT /identity/v3/projects/5df47d81e9ab4196abf4832a46739298/users/c80666a362864deab2e19c2d795311e8/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:08.425018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1c2b0-0746-4112-95a0-fe017494f4d5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:08.426281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1c2b0-0746-4112-95a0-fe017494f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:08.426676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1c2b0-0746-4112-95a0-fe017494f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:08.483270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fee1c2b0-0746-4112-95a0-fe017494f4d5 None None] MFA Rules not processed for user `c80666a362864deab2e19c2d795311e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:08.484772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fee1c2b0-0746-4112-95a0-fe017494f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:08.671879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 982/1964] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:08.684572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24054e24-b0a0-43c0-beff-c3df64d2007c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:08.686996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-24054e24-b0a0-43c0-beff-c3df64d2007c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:08.776270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24054e24-b0a0-43c0-beff-c3df64d2007c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:08.781054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1b2ad-da40-45ed-87af-c5d4ef02c05e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:08.782003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b2ad-da40-45ed-87af-c5d4ef02c05e service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:08.782534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b2ad-da40-45ed-87af-c5d4ef02c05e service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:08.782895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1b2ad-da40-45ed-87af-c5d4ef02c05e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:08.793237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1b2ad-da40-45ed-87af-c5d4ef02c05e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:08.794988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1b2ad-da40-45ed-87af-c5d4ef02c05e service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:08.805224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 983/1965] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:09.329982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1c2b0-0746-4112-95a0-fe017494f4d5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:09.331418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fee1c2b0-0746-4112-95a0-fe017494f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:09.343334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1c2b0-0746-4112-95a0-fe017494f4d5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:09.347031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778da7d9-948e-4dc7-813c-29d282699066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:09.347643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778da7d9-948e-4dc7-813c-29d282699066 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:09.347926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778da7d9-948e-4dc7-813c-29d282699066 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:09.348202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778da7d9-948e-4dc7-813c-29d282699066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:09.389615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:09.407203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778da7d9-948e-4dc7-813c-29d282699066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:09.408654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778da7d9-948e-4dc7-813c-29d282699066 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:09.418345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 983/1966] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:16.740840 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1b2ad-da40-45ed-87af-c5d4ef02c05e service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:16.750801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1b2ad-da40-45ed-87af-c5d4ef02c05e service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:16.754822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443689d-0ccc-4c8b-88e4-f83e889dc41a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:16.755694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443689d-0ccc-4c8b-88e4-f83e889dc41a service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:16.755694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443689d-0ccc-4c8b-88e4-f83e889dc41a service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:16.755884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443689d-0ccc-4c8b-88e4-f83e889dc41a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:16.841310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443689d-0ccc-4c8b-88e4-f83e889dc41a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:16.842731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443689d-0ccc-4c8b-88e4-f83e889dc41a service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:16.852174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 984/1967] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:19.414319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:24.654085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778da7d9-948e-4dc7-813c-29d282699066 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:24.716618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778da7d9-948e-4dc7-813c-29d282699066 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:24.721161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:24.721801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:24.722078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:24.722347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] PATH_INFO: `/v3/users/73773e48d2444895b0abacaaebba60c7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:24.723574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73773e48d2444895b0abacaaebba60c7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:24.726131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:24.775295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73773e48d2444895b0abacaaebba60c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:24.784277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73773e48d2444895b0abacaaebba60c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:24.784976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73773e48d2444895b0abacaaebba60c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:24.785586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73773e48d2444895b0abacaaebba60c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:24.799728 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73773e48d2444895b0abacaaebba60c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:24.814244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 984/1968] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:24 2022] DELETE /identity/v3/users/73773e48d2444895b0abacaaebba60c7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:24.819496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac957e-bba0-414b-94e7-c1cbf4aa0534 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:24.819963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac957e-bba0-414b-94e7-c1cbf4aa0534 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:24.820085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac957e-bba0-414b-94e7-c1cbf4aa0534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:24.840405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-28ac957e-bba0-414b-94e7-c1cbf4aa0534 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:24.841622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-28ac957e-bba0-414b-94e7-c1cbf4aa0534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:24.856209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:24.909775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 985/1969] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:24.920595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:24.922014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:24.988531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fc899-55f3-4990-94b9-b16dd134cf11 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:24.995667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898584b5-be09-41e8-bdac-ef51483cc723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:24.996576 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898584b5-be09-41e8-bdac-ef51483cc723 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:24.997274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898584b5-be09-41e8-bdac-ef51483cc723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:24.998136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898584b5-be09-41e8-bdac-ef51483cc723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:25.012135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898584b5-be09-41e8-bdac-ef51483cc723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:25.014686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898584b5-be09-41e8-bdac-ef51483cc723 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:25.032874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 985/1970] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:25.365711 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac957e-bba0-414b-94e7-c1cbf4aa0534 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:25.376884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-28ac957e-bba0-414b-94e7-c1cbf4aa0534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:25.398689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac957e-bba0-414b-94e7-c1cbf4aa0534 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:25.414645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:25.414645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:25.414645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:25.414645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] PATH_INFO: `/v3/projects/8307d1ce91cf4313b41be212733981a5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:25.415011 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8307d1ce91cf4313b41be212733981a5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:25.415011 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:25.485357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8307d1ce91cf4313b41be212733981a5', 'request_id': 'req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.496241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8307d1ce91cf4313b41be212733981a5', 'request_id': 'req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.504100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8307d1ce91cf4313b41be212733981a5', 'request_id': 'req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.519168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] The token cache is being invalidate because project 8307d1ce91cf4313b41be212733981a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:25.519462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.520231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.522493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 986/1971] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:25 2022] DELETE /identity/v3/projects/8307d1ce91cf4313b41be212733981a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:25.527376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898584b5-be09-41e8-bdac-ef51483cc723 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:25.538644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898584b5-be09-41e8-bdac-ef51483cc723 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:25.584537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:25.654729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:25.655465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:25.655834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:25.656177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] PATH_INFO: `/v3/users/9c45ba921c5f4cb38bda0a81ab11e425` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:25.657867 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c45ba921c5f4cb38bda0a81ab11e425)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:25.659918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:25.733048 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c45ba921c5f4cb38bda0a81ab11e425'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.745052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c45ba921c5f4cb38bda0a81ab11e425'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.748077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c45ba921c5f4cb38bda0a81ab11e425'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.749208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c45ba921c5f4cb38bda0a81ab11e425'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.776464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c45ba921c5f4cb38bda0a81ab11e425'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:25.805627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 986/1972] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:25 2022] DELETE /identity/v3/users/9c45ba921c5f4cb38bda0a81ab11e425 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:26.260324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:26.261606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:26.278570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c18ef3-953c-4dfc-8a60-ce7ba0cf6bfe admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:26.364579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:26.365354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:26.365699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:26.366030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] PATH_INFO: `/v3/projects/7cc1724a09884b3887fd83e1eba1fe1f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:26.367276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc1724a09884b3887fd83e1eba1fe1f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:26.369227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:26.418363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:26.420095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc1724a09884b3887fd83e1eba1fe1f', 'request_id': 'req-75ca3e50-bd9c-4362-b3f5-13622eb947a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.429527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc1724a09884b3887fd83e1eba1fe1f', 'request_id': 'req-75ca3e50-bd9c-4362-b3f5-13622eb947a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.436727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc1724a09884b3887fd83e1eba1fe1f', 'request_id': 'req-75ca3e50-bd9c-4362-b3f5-13622eb947a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.449684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] The token cache is being invalidate because project 7cc1724a09884b3887fd83e1eba1fe1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:26.450095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.450849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.452606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 987/1973] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:26 2022] DELETE /identity/v3/projects/7cc1724a09884b3887fd83e1eba1fe1f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:26.457228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:26.460234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:26.474209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcc49a-f64a-47d6-a36e-09cfcef6a2f6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:26.542547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:26.543232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:26.543516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:26.544734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] PATH_INFO: `/v3/users/c80666a362864deab2e19c2d795311e8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:26.545294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c80666a362864deab2e19c2d795311e8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:26.547776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:26.595604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:26.598510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c80666a362864deab2e19c2d795311e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.607289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c80666a362864deab2e19c2d795311e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.607978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c80666a362864deab2e19c2d795311e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.608608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c80666a362864deab2e19c2d795311e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.621357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c80666a362864deab2e19c2d795311e8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:26.635912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 987/1974] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:26 2022] DELETE /identity/v3/users/c80666a362864deab2e19c2d795311e8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:27.246963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:27.251993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:27.309097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3e50-bd9c-4362-b3f5-13622eb947a5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:27.390180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:27.391734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:27.401682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:27.403519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:27.404540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:27.404919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] PATH_INFO: `/v3/projects/5df47d81e9ab4196abf4832a46739298` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:27.407471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df47d81e9ab4196abf4832a46739298)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:27.410882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:27.507461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5689cd-5260-4c54-95ce-f263617b211f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:27.514229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df47d81e9ab4196abf4832a46739298', 'request_id': 'req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.527085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df47d81e9ab4196abf4832a46739298', 'request_id': 'req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.536085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df47d81e9ab4196abf4832a46739298', 'request_id': 'req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.555754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] The token cache is being invalidate because project 5df47d81e9ab4196abf4832a46739298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:27.556042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.557092 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.559538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 988/1975] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:27 2022] DELETE /identity/v3/projects/5df47d81e9ab4196abf4832a46739298 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:27.561161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:27.562675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:27.579208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:27.579980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:27.580337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:27.580688 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] PATH_INFO: `/v3/users/425ecb85e0ab46888481118ba400edba` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:27.581955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=425ecb85e0ab46888481118ba400edba)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:27.584247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:27.637013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425ecb85e0ab46888481118ba400edba'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.647483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425ecb85e0ab46888481118ba400edba'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.648464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425ecb85e0ab46888481118ba400edba'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.649150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425ecb85e0ab46888481118ba400edba'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.653018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6fe1a-1c26-4f6a-b7a2-4e9d336bad23 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:27.659262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:27.660309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:27.660591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:27.660906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] PATH_INFO: `/v3/users/36df16ba568648bf985c4782476672e7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:27.663447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] RBAC: Authorizing `identity:delete_user(user_id=36df16ba568648bf985c4782476672e7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:27.664740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:27.665640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:27.668515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-580021c4-4bf1-485b-b5a0-cb8385c8f088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425ecb85e0ab46888481118ba400edba'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.673372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:27.689646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 988/1976] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:27 2022] DELETE /identity/v3/users/425ecb85e0ab46888481118ba400edba => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:27.689646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8c953-aed8-4706-9ec3-1b24be984720 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:27.689646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8c953-aed8-4706-9ec3-1b24be984720 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:27.689646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8c953-aed8-4706-9ec3-1b24be984720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:27.711535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-01e8c953-aed8-4706-9ec3-1b24be984720 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:27.712776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-01e8c953-aed8-4706-9ec3-1b24be984720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:27.750721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36df16ba568648bf985c4782476672e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.760509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36df16ba568648bf985c4782476672e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.761264 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36df16ba568648bf985c4782476672e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.761896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36df16ba568648bf985c4782476672e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.777033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8d49ce1a-07d4-4309-8c04-32633c9d857c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36df16ba568648bf985c4782476672e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:27.790765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 989/1977] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:27.792316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3eae1c-2a2d-4c62-af49-e5b77dcd6fd8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:27.792608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3eae1c-2a2d-4c62-af49-e5b77dcd6fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:27.792881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3eae1c-2a2d-4c62-af49-e5b77dcd6fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:27.803523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 989/1978] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:27 2022] DELETE /identity/v3/users/36df16ba568648bf985c4782476672e7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:27.804859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975cd8c-5600-4f31-9a5d-d032fd8b9a5c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:27.805147 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975cd8c-5600-4f31-9a5d-d032fd8b9a5c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:27.805604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975cd8c-5600-4f31-9a5d-d032fd8b9a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:27.820168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7e3eae1c-2a2d-4c62-af49-e5b77dcd6fd8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:27.821870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e3eae1c-2a2d-4c62-af49-e5b77dcd6fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:27.829592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c975cd8c-5600-4f31-9a5d-d032fd8b9a5c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:27.831034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c975cd8c-5600-4f31-9a5d-d032fd8b9a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:27.905428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 990/1979] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:27.905428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193d88b-e98a-405b-9e26-1e84e32eb8e2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:27.905428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193d88b-e98a-405b-9e26-1e84e32eb8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:27.905428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193d88b-e98a-405b-9e26-1e84e32eb8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:27.931697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e193d88b-e98a-405b-9e26-1e84e32eb8e2 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:27.933393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e193d88b-e98a-405b-9e26-1e84e32eb8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:27.940884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 990/1980] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:27.944778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975cd8c-5600-4f31-9a5d-d032fd8b9a5c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:27.947954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c975cd8c-5600-4f31-9a5d-d032fd8b9a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:27.948141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 991/1981] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:27.951403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193d88b-e98a-405b-9e26-1e84e32eb8e2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:27.971691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193d88b-e98a-405b-9e26-1e84e32eb8e2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:27.979448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:27.980251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:27.980535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:27.980813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:27.981739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:27.982050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:27.984370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:28.002033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 992/1982] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:59:28.008533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:28.010797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:28.080575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975cd8c-5600-4f31-9a5d-d032fd8b9a5c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:28.088199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc7633-09e4-41f7-8b6e-0af706787264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:28.088958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7633-09e4-41f7-8b6e-0af706787264 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:28.089314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7633-09e4-41f7-8b6e-0af706787264 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:28.089663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc7633-09e4-41f7-8b6e-0af706787264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:28.108202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71309fa5-131c-45dd-adca-b797755c4e69 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:28.109427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc7633-09e4-41f7-8b6e-0af706787264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:28.110975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc7633-09e4-41f7-8b6e-0af706787264 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:28.117135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c19a8b-9bc6-4470-9a9f-50827e89637b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:28.117458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c19a8b-9bc6-4470-9a9f-50827e89637b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:28.118076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c19a8b-9bc6-4470-9a9f-50827e89637b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:28.118764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c19a8b-9bc6-4470-9a9f-50827e89637b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:28.122150 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 991/1983] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:28.122866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc7633-09e4-41f7-8b6e-0af706787264 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:28.133569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc7633-09e4-41f7-8b6e-0af706787264 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:28.135102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c19a8b-9bc6-4470-9a9f-50827e89637b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:28.136550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c19a8b-9bc6-4470-9a9f-50827e89637b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:28.138810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321c311-b6b4-4a8a-af34-acec0d723a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:28.139440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321c311-b6b4-4a8a-af34-acec0d723a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:28.139725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321c311-b6b4-4a8a-af34-acec0d723a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:28.140002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321c311-b6b4-4a8a-af34-acec0d723a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:28.140697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321c311-b6b4-4a8a-af34-acec0d723a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:28.142887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321c311-b6b4-4a8a-af34-acec0d723a22 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:28.155331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 993/1984] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:28.187091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 992/1985] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:28 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:28.191310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c19a8b-9bc6-4470-9a9f-50827e89637b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:28.203960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c19a8b-9bc6-4470-9a9f-50827e89637b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:28.225703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:28.342392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5515b3-cc1e-4fee-a302-28ab7b235298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:28.344324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5515b3-cc1e-4fee-a302-28ab7b235298 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:28.344711 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5515b3-cc1e-4fee-a302-28ab7b235298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:28.345091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5515b3-cc1e-4fee-a302-28ab7b235298 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:28.347117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5515b3-cc1e-4fee-a302-28ab7b235298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:28.349668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5515b3-cc1e-4fee-a302-28ab7b235298 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:28.392205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321c311-b6b4-4a8a-af34-acec0d723a22 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:28.393183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 994/1986] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:28 2022] POST /identity/v3/users => generated 291 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:28.409600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:28.420515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321c311-b6b4-4a8a-af34-acec0d723a22 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:28.428750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40248d07-aecc-46ba-a784-975e1493b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:28.429908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40248d07-aecc-46ba-a784-975e1493b24e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:28.435280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40248d07-aecc-46ba-a784-975e1493b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:28.435753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40248d07-aecc-46ba-a784-975e1493b24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:28.437442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40248d07-aecc-46ba-a784-975e1493b24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:28.440780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40248d07-aecc-46ba-a784-975e1493b24e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:28.454033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 993/1987] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:28.460908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5515b3-cc1e-4fee-a302-28ab7b235298 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:28.473387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5515b3-cc1e-4fee-a302-28ab7b235298 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:28.479005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb1ab7-0e13-4dfa-83ea-663cf82f5411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:28.481463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb1ab7-0e13-4dfa-83ea-663cf82f5411 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:28.481763 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb1ab7-0e13-4dfa-83ea-663cf82f5411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:28.482048 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb1ab7-0e13-4dfa-83ea-663cf82f5411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:28.482955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb1ab7-0e13-4dfa-83ea-663cf82f5411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:28.485917 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb1ab7-0e13-4dfa-83ea-663cf82f5411 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:28.512130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40248d07-aecc-46ba-a784-975e1493b24e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:28.521110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 995/1988] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:28.529517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40248d07-aecc-46ba-a784-975e1493b24e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:28.565641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5eea02-249d-42e2-969a-31a42cd8ebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:28.566476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5eea02-249d-42e2-969a-31a42cd8ebe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:28.566911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5eea02-249d-42e2-969a-31a42cd8ebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:28.567194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5eea02-249d-42e2-969a-31a42cd8ebe9 admin admin] PATH_INFO: `/v3/projects/d91c70c3df854f22952481cd1a77732c/users/4c379543bd2e458d85a856a3a88fa324/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:28.612955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5eea02-249d-42e2-969a-31a42cd8ebe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91c70c3df854f22952481cd1a77732c, user_id=4c379543bd2e458d85a856a3a88fa324, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:28.621589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5eea02-249d-42e2-969a-31a42cd8ebe9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:28.652059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 994/1989] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:28 2022] PUT /identity/v3/projects/d91c70c3df854f22952481cd1a77732c/users/4c379543bd2e458d85a856a3a88fa324/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:28.665453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a422c5f-f042-4acb-9028-76011fc2c44d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:28.665752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a422c5f-f042-4acb-9028-76011fc2c44d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:28.666201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a422c5f-f042-4acb-9028-76011fc2c44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:28.712715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4a422c5f-f042-4acb-9028-76011fc2c44d None None] MFA Rules not processed for user `4c379543bd2e458d85a856a3a88fa324`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:28.714689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a422c5f-f042-4acb-9028-76011fc2c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:28.874308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 996/1990] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:28.879720 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5eea02-249d-42e2-969a-31a42cd8ebe9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:28.927306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a422c5f-f042-4acb-9028-76011fc2c44d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:28.933344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a422c5f-f042-4acb-9028-76011fc2c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:29.040914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5eea02-249d-42e2-969a-31a42cd8ebe9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:29.047398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbfc70-ed00-41bb-a0ca-b5d9dd3c8dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:29.048112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbfc70-ed00-41bb-a0ca-b5d9dd3c8dae admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.048458 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbfc70-ed00-41bb-a0ca-b5d9dd3c8dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.048796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbfc70-ed00-41bb-a0ca-b5d9dd3c8dae admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.049530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbfc70-ed00-41bb-a0ca-b5d9dd3c8dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:29.053671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbfc70-ed00-41bb-a0ca-b5d9dd3c8dae admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:29.112380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 995/1991] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:28 2022] POST /identity/v3/projects => generated 383 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:29.116355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbfc70-ed00-41bb-a0ca-b5d9dd3c8dae admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:29.126246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a422c5f-f042-4acb-9028-76011fc2c44d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:29.157021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:29.223074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:29.224518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.224863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.225141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] PATH_INFO: `/v3/projects/4d3dd8c4c5e84003801865b3cb7afd5c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.226871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d3dd8c4c5e84003801865b3cb7afd5c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:29.230411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:29.281121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbfc70-ed00-41bb-a0ca-b5d9dd3c8dae admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:29.288193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907278a-1457-42cd-95f9-e75f414e3900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:29.289056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907278a-1457-42cd-95f9-e75f414e3900 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.289530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907278a-1457-42cd-95f9-e75f414e3900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.290356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907278a-1457-42cd-95f9-e75f414e3900 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.291397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907278a-1457-42cd-95f9-e75f414e3900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:29.292881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907278a-1457-42cd-95f9-e75f414e3900 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:29.322680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d3dd8c4c5e84003801865b3cb7afd5c', 'request_id': 'req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.339909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 996/1992] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:29 2022] POST /identity/v3/users => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:29.341025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907278a-1457-42cd-95f9-e75f414e3900 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:29.343419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e907278a-1457-42cd-95f9-e75f414e3900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:29.352734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d3dd8c4c5e84003801865b3cb7afd5c', 'request_id': 'req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.358622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907278a-1457-42cd-95f9-e75f414e3900 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:29.376802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d3dd8c4c5e84003801865b3cb7afd5c', 'request_id': 'req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.397914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] The token cache is being invalidate because project 4d3dd8c4c5e84003801865b3cb7afd5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:29.398607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.399300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.402311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 997/1993] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:28 2022] DELETE /identity/v3/projects/4d3dd8c4c5e84003801865b3cb7afd5c => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:29.406705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:29.406705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:29.433980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:29.457752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:29.458258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.458567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.458849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] PATH_INFO: `/v3/projects/0ecccdebfe9e4cf29cc1f6d2b21f7247` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.460054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ecccdebfe9e4cf29cc1f6d2b21f7247)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:29.461986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:29.502216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c11a2-4bb6-4910-ac44-ec51bbdcefce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:29.509317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81d95c-91dd-41c8-a6b7-b8dd5091f328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:29.510084 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81d95c-91dd-41c8-a6b7-b8dd5091f328 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.510454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81d95c-91dd-41c8-a6b7-b8dd5091f328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.510786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81d95c-91dd-41c8-a6b7-b8dd5091f328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.511592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81d95c-91dd-41c8-a6b7-b8dd5091f328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:29.514276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81d95c-91dd-41c8-a6b7-b8dd5091f328 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:29.525680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 998/1994] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:29.527463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d87833-79df-47f5-b566-eea2c1dbfceb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.527463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d87833-79df-47f5-b566-eea2c1dbfceb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.527670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d87833-79df-47f5-b566-eea2c1dbfceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.555887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ecccdebfe9e4cf29cc1f6d2b21f7247', 'request_id': 'req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.565176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b4d87833-79df-47f5-b566-eea2c1dbfceb None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:29.566524 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4d87833-79df-47f5-b566-eea2c1dbfceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:29.571646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ecccdebfe9e4cf29cc1f6d2b21f7247', 'request_id': 'req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.578108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ecccdebfe9e4cf29cc1f6d2b21f7247', 'request_id': 'req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.599052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] The token cache is being invalidate because project 0ecccdebfe9e4cf29cc1f6d2b21f7247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:29.600939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.605379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.607427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 997/1995] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:29 2022] DELETE /identity/v3/projects/0ecccdebfe9e4cf29cc1f6d2b21f7247 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:29.608299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:29.610115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:29.671013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:29.690456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 999/1996] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:29.691371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d87833-79df-47f5-b566-eea2c1dbfceb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:29.694096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4d87833-79df-47f5-b566-eea2c1dbfceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:29.709164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d87833-79df-47f5-b566-eea2c1dbfceb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:29.714029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:29.714719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.715134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.715419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] PATH_INFO: `/v3/users/87cc4f7a4f2647cfb08e742b9db00cec` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.717544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87cc4f7a4f2647cfb08e742b9db00cec)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:29.719918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:29.720946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507eb1cc-fa47-4f59-aa48-bc7b80e2035e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:29.725996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48328d99-9554-4520-89a4-f4ea6697323c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:29.727482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48328d99-9554-4520-89a4-f4ea6697323c admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.727482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48328d99-9554-4520-89a4-f4ea6697323c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.727482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48328d99-9554-4520-89a4-f4ea6697323c admin admin] PATH_INFO: `/v3/projects/68edc9cc46d643b1ba8a4ce42971218c/users/f3cda8cb71a64ef2a727fd66d94b3641/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.747154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48328d99-9554-4520-89a4-f4ea6697323c admin admin] RBAC: Authorizing `identity:create_grant(project_id=68edc9cc46d643b1ba8a4ce42971218c, user_id=f3cda8cb71a64ef2a727fd66d94b3641, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:29.750054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48328d99-9554-4520-89a4-f4ea6697323c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:29.771068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 998/1997] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:29 2022] PUT /identity/v3/projects/68edc9cc46d643b1ba8a4ce42971218c/users/f3cda8cb71a64ef2a727fd66d94b3641/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:29.772918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562f81d-15b3-43ab-9047-1c213c0ebc7c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.773213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562f81d-15b3-43ab-9047-1c213c0ebc7c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.773536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562f81d-15b3-43ab-9047-1c213c0ebc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.800455 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0562f81d-15b3-43ab-9047-1c213c0ebc7c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:29.800689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87cc4f7a4f2647cfb08e742b9db00cec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.802247 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0562f81d-15b3-43ab-9047-1c213c0ebc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:29.810995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87cc4f7a4f2647cfb08e742b9db00cec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.813709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87cc4f7a4f2647cfb08e742b9db00cec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.814368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87cc4f7a4f2647cfb08e742b9db00cec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.832660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87cc4f7a4f2647cfb08e742b9db00cec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:29.856519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1000/1998] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:29 2022] DELETE /identity/v3/users/87cc4f7a4f2647cfb08e742b9db00cec => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:29.861165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:29.862819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:29.888074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:29.921532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 999/1999] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:29.922790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562f81d-15b3-43ab-9047-1c213c0ebc7c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:29.942101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562f81d-15b3-43ab-9047-1c213c0ebc7c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:29.960909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8240c77-f186-4cf0-8f73-984d3b9b9413 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:29.966577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645f12b-913a-435c-b935-c8bc7bef95bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:29.967562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645f12b-913a-435c-b935-c8bc7bef95bb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:29.967946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645f12b-913a-435c-b935-c8bc7bef95bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:29.968326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645f12b-913a-435c-b935-c8bc7bef95bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:29.969141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645f12b-913a-435c-b935-c8bc7bef95bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:29.971095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645f12b-913a-435c-b935-c8bc7bef95bb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:29.991765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1001/2000] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:29.996589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645f12b-913a-435c-b935-c8bc7bef95bb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.039127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645f12b-913a-435c-b935-c8bc7bef95bb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:30.047885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f6a51-bf21-44a5-be5b-09a71a499b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.048607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f6a51-bf21-44a5-be5b-09a71a499b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.048916 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f6a51-bf21-44a5-be5b-09a71a499b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.049198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f6a51-bf21-44a5-be5b-09a71a499b43 admin admin] PATH_INFO: `/v3/projects/68edc9cc46d643b1ba8a4ce42971218c/users/f3cda8cb71a64ef2a727fd66d94b3641/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.052729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f6a51-bf21-44a5-be5b-09a71a499b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68edc9cc46d643b1ba8a4ce42971218c, user_id=f3cda8cb71a64ef2a727fd66d94b3641, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:30.057313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f6a51-bf21-44a5-be5b-09a71a499b43 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:30.084527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1002/2001] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:29 2022] PUT /identity/v3/projects/68edc9cc46d643b1ba8a4ce42971218c/users/f3cda8cb71a64ef2a727fd66d94b3641/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:30.086605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd85dbe-4f34-4677-96a1-cf99e4ea4eb9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.086935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd85dbe-4f34-4677-96a1-cf99e4ea4eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.087378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd85dbe-4f34-4677-96a1-cf99e4ea4eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.120991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1714e-0889-4593-9663-ca71de29848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.122602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1714e-0889-4593-9663-ca71de29848f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.123467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1714e-0889-4593-9663-ca71de29848f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.123846 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3fd85dbe-4f34-4677-96a1-cf99e4ea4eb9 None None] MFA Rules not processed for user `f3cda8cb71a64ef2a727fd66d94b3641`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:30.125117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3fd85dbe-4f34-4677-96a1-cf99e4ea4eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:30.125569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1714e-0889-4593-9663-ca71de29848f admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.127273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1714e-0889-4593-9663-ca71de29848f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:30.130733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1714e-0889-4593-9663-ca71de29848f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:30.137485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1714e-0889-4593-9663-ca71de29848f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:30.157240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1000/2002] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:59:30.158597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1714e-0889-4593-9663-ca71de29848f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.181397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1714e-0889-4593-9663-ca71de29848f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:30.229416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:30.287288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e60d98-f8b7-4f85-8a32-78650d1b04a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.288035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e60d98-f8b7-4f85-8a32-78650d1b04a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.288420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e60d98-f8b7-4f85-8a32-78650d1b04a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.288568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e60d98-f8b7-4f85-8a32-78650d1b04a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.291122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e60d98-f8b7-4f85-8a32-78650d1b04a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:30.293064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e60d98-f8b7-4f85-8a32-78650d1b04a2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:30.309261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1003/2003] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:30.321272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd85dbe-4f34-4677-96a1-cf99e4ea4eb9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.322855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3fd85dbe-4f34-4677-96a1-cf99e4ea4eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:30.353442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1001/2004] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:30 2022] POST /identity/v3/projects => generated 371 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:30.363525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e60d98-f8b7-4f85-8a32-78650d1b04a2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.406578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd85dbe-4f34-4677-96a1-cf99e4ea4eb9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:30.476384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e60d98-f8b7-4f85-8a32-78650d1b04a2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:30.483759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58513c9d-36a8-4f05-8c5b-c37793f9cb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.484491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58513c9d-36a8-4f05-8c5b-c37793f9cb7f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.484911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58513c9d-36a8-4f05-8c5b-c37793f9cb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.485194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58513c9d-36a8-4f05-8c5b-c37793f9cb7f admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.486011 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58513c9d-36a8-4f05-8c5b-c37793f9cb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:30.488340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58513c9d-36a8-4f05-8c5b-c37793f9cb7f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:30.494210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d38591-9c1c-4c2e-97f0-6d2f069be2ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.495324 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d38591-9c1c-4c2e-97f0-6d2f069be2ec service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.495984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d38591-9c1c-4c2e-97f0-6d2f069be2ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.496465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d38591-9c1c-4c2e-97f0-6d2f069be2ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.522276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1002/2005] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:30 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:30.523376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58513c9d-36a8-4f05-8c5b-c37793f9cb7f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.530733 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:30.535205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58513c9d-36a8-4f05-8c5b-c37793f9cb7f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:30.544923 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e78e-b5a9-42f3-b517-7049b0f41cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.545853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e78e-b5a9-42f3-b517-7049b0f41cca admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.546212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e78e-b5a9-42f3-b517-7049b0f41cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.546581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e78e-b5a9-42f3-b517-7049b0f41cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.547621 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3e78e-b5a9-42f3-b517-7049b0f41cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:30.549603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3e78e-b5a9-42f3-b517-7049b0f41cca admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:30.565433 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1003/2006] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:30.566591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e78e-b5a9-42f3-b517-7049b0f41cca admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.580041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e78e-b5a9-42f3-b517-7049b0f41cca admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:30.587123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524823b5-2f39-4a82-a885-eb868440ed3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.588146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524823b5-2f39-4a82-a885-eb868440ed3b admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.588541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524823b5-2f39-4a82-a885-eb868440ed3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.589116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524823b5-2f39-4a82-a885-eb868440ed3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.590000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524823b5-2f39-4a82-a885-eb868440ed3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:30.591789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524823b5-2f39-4a82-a885-eb868440ed3b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:30.595531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d38591-9c1c-4c2e-97f0-6d2f069be2ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:30.597266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d38591-9c1c-4c2e-97f0-6d2f069be2ec service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:30.611690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1004/2007] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:30.613382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524823b5-2f39-4a82-a885-eb868440ed3b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.615709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-524823b5-2f39-4a82-a885-eb868440ed3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:30.634254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1004/2008] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:30.637427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d38591-9c1c-4c2e-97f0-6d2f069be2ec service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.649865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524823b5-2f39-4a82-a885-eb868440ed3b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:30.657052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.657722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.658003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.658572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] PATH_INFO: `/v3/projects/1c38378b250a4014b700eb2863c96df7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.658945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d38591-9c1c-4c2e-97f0-6d2f069be2ec service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:30.659873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c38378b250a4014b700eb2863c96df7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:30.663475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:30.668977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cadc6c-46fa-4f27-ae89-7a8dab480f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.669670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cadc6c-46fa-4f27-ae89-7a8dab480f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.669949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cadc6c-46fa-4f27-ae89-7a8dab480f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.670220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cadc6c-46fa-4f27-ae89-7a8dab480f3e admin admin] PATH_INFO: `/v3/projects/76df23454a3c4a9bb4f61b044ed5a426/users/b1a98edf8e5a4a22b3ae7d041b582b2d/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.706900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cadc6c-46fa-4f27-ae89-7a8dab480f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=76df23454a3c4a9bb4f61b044ed5a426, user_id=b1a98edf8e5a4a22b3ae7d041b582b2d, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:30.713713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cadc6c-46fa-4f27-ae89-7a8dab480f3e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:30.750168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1005/2009] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:30 2022] PUT /identity/v3/projects/76df23454a3c4a9bb4f61b044ed5a426/users/b1a98edf8e5a4a22b3ae7d041b582b2d/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:30.756119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cadc6c-46fa-4f27-ae89-7a8dab480f3e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.769942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c38378b250a4014b700eb2863c96df7', 'request_id': 'req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:30.780071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c38378b250a4014b700eb2863c96df7', 'request_id': 'req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:30.782899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:30.790366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c38378b250a4014b700eb2863c96df7', 'request_id': 'req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:30.809351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] The token cache is being invalidate because project 1c38378b250a4014b700eb2863c96df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:30.810378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:30.811350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a62629cd-4ab4-467b-a055-5f7cd1c3e8de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:30.813595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1005/2010] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:30 2022] DELETE /identity/v3/projects/1c38378b250a4014b700eb2863c96df7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:30.815200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cccabd-878f-4b91-8fef-4f1b79de89ce None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.815726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cccabd-878f-4b91-8fef-4f1b79de89ce None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.816629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cccabd-878f-4b91-8fef-4f1b79de89ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.842930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-66cccabd-878f-4b91-8fef-4f1b79de89ce None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:30.844464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-66cccabd-878f-4b91-8fef-4f1b79de89ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:30.867563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cadc6c-46fa-4f27-ae89-7a8dab480f3e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:30.920990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1006/2011] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:30.922927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42ffb8-7abb-4b6d-ab17-d2b924e84c17 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.923297 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42ffb8-7abb-4b6d-ab17-d2b924e84c17 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.923655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42ffb8-7abb-4b6d-ab17-d2b924e84c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.947585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8d42ffb8-7abb-4b6d-ab17-d2b924e84c17 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:30.948942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8d42ffb8-7abb-4b6d-ab17-d2b924e84c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:30.960844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1007/2012] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:30.962003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42ffb8-7abb-4b6d-ab17-d2b924e84c17 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:30.963619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8d42ffb8-7abb-4b6d-ab17-d2b924e84c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:30.968855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508a1e2-2f25-433b-bb48-0f83b2a17ce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:30.969905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508a1e2-2f25-433b-bb48-0f83b2a17ce8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:30.970395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508a1e2-2f25-433b-bb48-0f83b2a17ce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:30.970904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508a1e2-2f25-433b-bb48-0f83b2a17ce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:30.973230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9508a1e2-2f25-433b-bb48-0f83b2a17ce8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:31.051876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508a1e2-2f25-433b-bb48-0f83b2a17ce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.053650 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508a1e2-2f25-433b-bb48-0f83b2a17ce8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.053920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42ffb8-7abb-4b6d-ab17-d2b924e84c17 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.058274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f87a-7109-4c21-9fde-6dd849e7c90a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.059135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f87a-7109-4c21-9fde-6dd849e7c90a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.059487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f87a-7109-4c21-9fde-6dd849e7c90a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.059835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f87a-7109-4c21-9fde-6dd849e7c90a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.065882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1006/2013] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:31.068394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd7880-c9fa-43a1-9ab5-34a100de3568 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.069198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd7880-c9fa-43a1-9ab5-34a100de3568 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.069685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd7880-c9fa-43a1-9ab5-34a100de3568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.094961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d6fd7880-c9fa-43a1-9ab5-34a100de3568 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:31.096595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d6fd7880-c9fa-43a1-9ab5-34a100de3568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:31.097236 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05f87a-7109-4c21-9fde-6dd849e7c90a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.099723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05f87a-7109-4c21-9fde-6dd849e7c90a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.110960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1007/2014] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:31.115235 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd7880-c9fa-43a1-9ab5-34a100de3568 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.118222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1008/2015] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:31.141979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd7880-c9fa-43a1-9ab5-34a100de3568 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.148228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde72a80-1c23-4e81-8f56-c25e322ad91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.148919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde72a80-1c23-4e81-8f56-c25e322ad91d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.149449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde72a80-1c23-4e81-8f56-c25e322ad91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.150349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde72a80-1c23-4e81-8f56-c25e322ad91d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.151153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde72a80-1c23-4e81-8f56-c25e322ad91d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:31.151460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde72a80-1c23-4e81-8f56-c25e322ad91d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.153507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde72a80-1c23-4e81-8f56-c25e322ad91d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.182752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1008/2016] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:59:31.187403 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f87a-7109-4c21-9fde-6dd849e7c90a service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.195278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:31.211304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f87a-7109-4c21-9fde-6dd849e7c90a service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.230603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40732f-a97a-47b5-8e11-ede28b9c51eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.231250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40732f-a97a-47b5-8e11-ede28b9c51eb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.231532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40732f-a97a-47b5-8e11-ede28b9c51eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.231805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40732f-a97a-47b5-8e11-ede28b9c51eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.232480 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40732f-a97a-47b5-8e11-ede28b9c51eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.234416 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40732f-a97a-47b5-8e11-ede28b9c51eb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.297623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1009/2017] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:31 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:31.298848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde72a80-1c23-4e81-8f56-c25e322ad91d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.310480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde72a80-1c23-4e81-8f56-c25e322ad91d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.448231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189b69c-8473-4ea0-bf61-19392ae6d923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.448866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189b69c-8473-4ea0-bf61-19392ae6d923 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.449147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189b69c-8473-4ea0-bf61-19392ae6d923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.449436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189b69c-8473-4ea0-bf61-19392ae6d923 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.450597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189b69c-8473-4ea0-bf61-19392ae6d923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.453445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189b69c-8473-4ea0-bf61-19392ae6d923 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.471471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:31.491422 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1009/2018] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:31 2022] POST /identity/v3/users => generated 282 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:31.495575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40732f-a97a-47b5-8e11-ede28b9c51eb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.523387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40732f-a97a-47b5-8e11-ede28b9c51eb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.532700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189b69c-8473-4ea0-bf61-19392ae6d923 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.532856 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673c499-7c9b-4692-8d6e-c6e907a3978e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.533807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673c499-7c9b-4692-8d6e-c6e907a3978e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.534289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673c499-7c9b-4692-8d6e-c6e907a3978e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.534415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7189b69c-8473-4ea0-bf61-19392ae6d923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:31.535036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673c499-7c9b-4692-8d6e-c6e907a3978e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.536028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673c499-7c9b-4692-8d6e-c6e907a3978e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.537844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673c499-7c9b-4692-8d6e-c6e907a3978e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.548317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189b69c-8473-4ea0-bf61-19392ae6d923 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.552158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1010/2019] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:31.556296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.556796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673c499-7c9b-4692-8d6e-c6e907a3978e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.557065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.558345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.558726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] PATH_INFO: `/v3/users/4c379543bd2e458d85a856a3a88fa324` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.560204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c379543bd2e458d85a856a3a88fa324)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.562394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.587976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673c499-7c9b-4692-8d6e-c6e907a3978e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.603818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee36e-b539-4aa8-bfc4-39f79b032378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.604491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee36e-b539-4aa8-bfc4-39f79b032378 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.604801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee36e-b539-4aa8-bfc4-39f79b032378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.605072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ee36e-b539-4aa8-bfc4-39f79b032378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.606085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ee36e-b539-4aa8-bfc4-39f79b032378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.608111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ee36e-b539-4aa8-bfc4-39f79b032378 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.623663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1011/2020] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:31.628584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee36e-b539-4aa8-bfc4-39f79b032378 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.648660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ee36e-b539-4aa8-bfc4-39f79b032378 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.649905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c379543bd2e458d85a856a3a88fa324'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:31.654897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929c24a-edea-4625-b74c-2e909b2bca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.655906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929c24a-edea-4625-b74c-2e909b2bca1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.656518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929c24a-edea-4625-b74c-2e909b2bca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.656990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929c24a-edea-4625-b74c-2e909b2bca1b admin admin] PATH_INFO: `/v3/projects/b7ca37f9b98d4ae4a4edae28c1f8d13d/users/52099395f92347bcabf3c0916d4f283e/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.667413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c379543bd2e458d85a856a3a88fa324'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:31.669170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c379543bd2e458d85a856a3a88fa324'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:31.672215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c379543bd2e458d85a856a3a88fa324'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:31.677970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929c24a-edea-4625-b74c-2e909b2bca1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ca37f9b98d4ae4a4edae28c1f8d13d, user_id=52099395f92347bcabf3c0916d4f283e, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.682381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929c24a-edea-4625-b74c-2e909b2bca1b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.695120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-06e03454-bc28-426d-bf45-1953f2dbb25a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c379543bd2e458d85a856a3a88fa324'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:31.737234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1010/2021] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:31 2022] DELETE /identity/v3/users/4c379543bd2e458d85a856a3a88fa324 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:31.742481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1012/2022] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:31 2022] PUT /identity/v3/projects/b7ca37f9b98d4ae4a4edae28c1f8d13d/users/52099395f92347bcabf3c0916d4f283e/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:31.743115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9e6fb-cc68-4369-bf9f-3a150d0c84f9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.743421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9e6fb-cc68-4369-bf9f-3a150d0c84f9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.743975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9e6fb-cc68-4369-bf9f-3a150d0c84f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.754664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61238c-892b-44f3-9dc4-8d6350856a56 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.755672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61238c-892b-44f3-9dc4-8d6350856a56 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.755957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61238c-892b-44f3-9dc4-8d6350856a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.773598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6ff9e6fb-cc68-4369-bf9f-3a150d0c84f9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:31.775380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ff9e6fb-cc68-4369-bf9f-3a150d0c84f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:31.794090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4d61238c-892b-44f3-9dc4-8d6350856a56 None None] MFA Rules not processed for user `52099395f92347bcabf3c0916d4f283e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:31.795567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d61238c-892b-44f3-9dc4-8d6350856a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:31.866638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1011/2023] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:31.876599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9e6fb-cc68-4369-bf9f-3a150d0c84f9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.880192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1013/2024] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:31.880503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ff9e6fb-cc68-4369-bf9f-3a150d0c84f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:31.886749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61238c-892b-44f3-9dc4-8d6350856a56 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.888636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d61238c-892b-44f3-9dc4-8d6350856a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:31.907629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61238c-892b-44f3-9dc4-8d6350856a56 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.917634 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07522e03-8772-4bb7-a3a3-fdde89c1bf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.918259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07522e03-8772-4bb7-a3a3-fdde89c1bf3d admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.918883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07522e03-8772-4bb7-a3a3-fdde89c1bf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.919312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07522e03-8772-4bb7-a3a3-fdde89c1bf3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.919995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07522e03-8772-4bb7-a3a3-fdde89c1bf3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.923539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07522e03-8772-4bb7-a3a3-fdde89c1bf3d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:31.967054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9e6fb-cc68-4369-bf9f-3a150d0c84f9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:31.972111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce771f94-3919-4576-8dd1-25444bc7a493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:31.973174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce771f94-3919-4576-8dd1-25444bc7a493 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:31.974631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce771f94-3919-4576-8dd1-25444bc7a493 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:31.975421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce771f94-3919-4576-8dd1-25444bc7a493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:31.980424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1014/2025] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:31 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:31.981579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07522e03-8772-4bb7-a3a3-fdde89c1bf3d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:31.995158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce771f94-3919-4576-8dd1-25444bc7a493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:31.997095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce771f94-3919-4576-8dd1-25444bc7a493 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:32.009400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1012/2026] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:32.119878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07522e03-8772-4bb7-a3a3-fdde89c1bf3d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:32.125309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e4ce3-da0b-4c3e-ab96-30db207acd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:32.126211 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e4ce3-da0b-4c3e-ab96-30db207acd43 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:32.126688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e4ce3-da0b-4c3e-ab96-30db207acd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:32.127039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e4ce3-da0b-4c3e-ab96-30db207acd43 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:32.128118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e4ce3-da0b-4c3e-ab96-30db207acd43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:32.131346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e4ce3-da0b-4c3e-ab96-30db207acd43 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:32.160373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:32.182337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1015/2027] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:31 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:32.186547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce771f94-3919-4576-8dd1-25444bc7a493 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:32.211151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce771f94-3919-4576-8dd1-25444bc7a493 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:32.217187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7d6c2-8928-4136-92a6-f1f967ab92d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:32.218060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7d6c2-8928-4136-92a6-f1f967ab92d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:32.218702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7d6c2-8928-4136-92a6-f1f967ab92d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:32.219569 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7d6c2-8928-4136-92a6-f1f967ab92d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:32.220665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7d6c2-8928-4136-92a6-f1f967ab92d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:32.228059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7d6c2-8928-4136-92a6-f1f967ab92d7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:32.235235 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:32.256948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1013/2028] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:32.258835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e4ce3-da0b-4c3e-ab96-30db207acd43 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:32.282783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e4ce3-da0b-4c3e-ab96-30db207acd43 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:32.297359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f39b2-4a7d-40b7-b459-0ed108687bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:32.298120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f39b2-4a7d-40b7-b459-0ed108687bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:32.298402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f39b2-4a7d-40b7-b459-0ed108687bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:32.298727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f39b2-4a7d-40b7-b459-0ed108687bdc admin admin] PATH_INFO: `/v3/projects/d992033017e04e79b2e43ad210e881f9/users/3bdc66ed1d9b48368dba96967e914894/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:32.362391 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f39b2-4a7d-40b7-b459-0ed108687bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d992033017e04e79b2e43ad210e881f9, user_id=3bdc66ed1d9b48368dba96967e914894, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:32.371089 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f39b2-4a7d-40b7-b459-0ed108687bdc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:32.412647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1016/2029] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:32 2022] PUT /identity/v3/projects/d992033017e04e79b2e43ad210e881f9/users/3bdc66ed1d9b48368dba96967e914894/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:32.413935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7d6c2-8928-4136-92a6-f1f967ab92d7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:32.424906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7d6c2-8928-4136-92a6-f1f967ab92d7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:32.553766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412db672-d164-4fe8-98c3-f2740f5bcb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:32.555430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412db672-d164-4fe8-98c3-f2740f5bcb38 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:32.555917 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412db672-d164-4fe8-98c3-f2740f5bcb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:32.556433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412db672-d164-4fe8-98c3-f2740f5bcb38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:32.558972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412db672-d164-4fe8-98c3-f2740f5bcb38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:32.561088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412db672-d164-4fe8-98c3-f2740f5bcb38 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:32.580359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1014/2030] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:32.584674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f39b2-4a7d-40b7-b459-0ed108687bdc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:32.603611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f39b2-4a7d-40b7-b459-0ed108687bdc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:32.615796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d33e5c-83f6-4990-8aef-17f1c34b8700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:32.616383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33e5c-83f6-4990-8aef-17f1c34b8700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:32.619710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33e5c-83f6-4990-8aef-17f1c34b8700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:32.620232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33e5c-83f6-4990-8aef-17f1c34b8700 admin admin] PATH_INFO: `/v3/projects/d992033017e04e79b2e43ad210e881f9/users/3bdc66ed1d9b48368dba96967e914894/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:32.623105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d33e5c-83f6-4990-8aef-17f1c34b8700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d992033017e04e79b2e43ad210e881f9, user_id=3bdc66ed1d9b48368dba96967e914894, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:32.626662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d33e5c-83f6-4990-8aef-17f1c34b8700 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:32.653339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412db672-d164-4fe8-98c3-f2740f5bcb38 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:32.655820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-412db672-d164-4fe8-98c3-f2740f5bcb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:32.673351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1017/2031] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:32 2022] PUT /identity/v3/projects/d992033017e04e79b2e43ad210e881f9/users/3bdc66ed1d9b48368dba96967e914894/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:32.677639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412db672-d164-4fe8-98c3-f2740f5bcb38 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:32.683573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e26928-3224-4030-9300-e194174190f2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:32.683856 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e26928-3224-4030-9300-e194174190f2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:32.684137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e26928-3224-4030-9300-e194174190f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:32.773803 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a5e26928-3224-4030-9300-e194174190f2 None None] MFA Rules not processed for user `3bdc66ed1d9b48368dba96967e914894`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:32.775058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a5e26928-3224-4030-9300-e194174190f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:32.867595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:32.868285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:32.868633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:32.868844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] PATH_INFO: `/v3/projects/d91c70c3df854f22952481cd1a77732c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:32.871471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d91c70c3df854f22952481cd1a77732c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:32.873452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:32.942596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1018/2032] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:32.950566 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e26928-3224-4030-9300-e194174190f2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:32.952415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a5e26928-3224-4030-9300-e194174190f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:32.998590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd91c70c3df854f22952481cd1a77732c', 'request_id': 'req-82e5309d-2d22-40c5-ab8a-bc001545da8b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.015662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd91c70c3df854f22952481cd1a77732c', 'request_id': 'req-82e5309d-2d22-40c5-ab8a-bc001545da8b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.032603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd91c70c3df854f22952481cd1a77732c', 'request_id': 'req-82e5309d-2d22-40c5-ab8a-bc001545da8b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.050090 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] The token cache is being invalidate because project d91c70c3df854f22952481cd1a77732c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:33.050752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.052560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.057844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1015/2033] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:32 2022] DELETE /identity/v3/projects/d91c70c3df854f22952481cd1a77732c => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:33.061659 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:33.065761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:33.072667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e26928-3224-4030-9300-e194174190f2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:33.129377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:33.180379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62132eb7-74b3-4c6a-bd6a-aa116eaff569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:33.181350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62132eb7-74b3-4c6a-bd6a-aa116eaff569 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:33.181712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62132eb7-74b3-4c6a-bd6a-aa116eaff569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:33.182038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62132eb7-74b3-4c6a-bd6a-aa116eaff569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:33.186926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-62132eb7-74b3-4c6a-bd6a-aa116eaff569 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:33.194155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5309d-2d22-40c5-ab8a-bc001545da8b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:33.200219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:33.201346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:33.202306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:33.202925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] PATH_INFO: `/v3/users/f3cda8cb71a64ef2a727fd66d94b3641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:33.205118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3cda8cb71a64ef2a727fd66d94b3641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:33.207139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:33.208550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:33.308347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3cda8cb71a64ef2a727fd66d94b3641'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.322338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3cda8cb71a64ef2a727fd66d94b3641'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.323151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3cda8cb71a64ef2a727fd66d94b3641'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.324142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3cda8cb71a64ef2a727fd66d94b3641'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.342913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3cda8cb71a64ef2a727fd66d94b3641'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:33.361163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1016/2034] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:33 2022] DELETE /identity/v3/users/f3cda8cb71a64ef2a727fd66d94b3641 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:33.374037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62132eb7-74b3-4c6a-bd6a-aa116eaff569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:33.375870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62132eb7-74b3-4c6a-bd6a-aa116eaff569 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:33.396747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1019/2035] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:59:33.569990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:33.569990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:33.673964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72cebe-d256-493e-84d3-c14bdf4c72b2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:33.678864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8732364-4085-40e0-b68a-c001ca337df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:33.679587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8732364-4085-40e0-b68a-c001ca337df8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:33.679939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8732364-4085-40e0-b68a-c001ca337df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:33.680282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8732364-4085-40e0-b68a-c001ca337df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:33.682325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e8732364-4085-40e0-b68a-c001ca337df8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:33.765398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8732364-4085-40e0-b68a-c001ca337df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:33.766894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8732364-4085-40e0-b68a-c001ca337df8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:33.772482 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:33.777734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1017/2036] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:33 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:59:33.800805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62132eb7-74b3-4c6a-bd6a-aa116eaff569 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:33.827530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62132eb7-74b3-4c6a-bd6a-aa116eaff569 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:33.914016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:33.914785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:33.915071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:33.915332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] PATH_INFO: `/v3/projects/68edc9cc46d643b1ba8a4ce42971218c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:33.916838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=68edc9cc46d643b1ba8a4ce42971218c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:33.919068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:33.998385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68edc9cc46d643b1ba8a4ce42971218c', 'request_id': 'req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:34.010127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68edc9cc46d643b1ba8a4ce42971218c', 'request_id': 'req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:34.016215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68edc9cc46d643b1ba8a4ce42971218c', 'request_id': 'req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:34.033162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] The token cache is being invalidate because project 68edc9cc46d643b1ba8a4ce42971218c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:34.033453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:34.034909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-25cc5a5b-070d-42f8-92cb-0cc3f08211ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:34.037646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1020/2037] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:33 2022] DELETE /identity/v3/projects/68edc9cc46d643b1ba8a4ce42971218c => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:34.042739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44aab9-e29b-484e-b1af-87ccdc1fe1b6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:34.043098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44aab9-e29b-484e-b1af-87ccdc1fe1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:34.043451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44aab9-e29b-484e-b1af-87ccdc1fe1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:34.065297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9b44aab9-e29b-484e-b1af-87ccdc1fe1b6 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:34.066624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b44aab9-e29b-484e-b1af-87ccdc1fe1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:34.138229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1018/2038] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:59:34.146063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4cd9-b50d-467d-bc92-1ee686f1cadc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:34.146417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4cd9-b50d-467d-bc92-1ee686f1cadc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:34.146808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4cd9-b50d-467d-bc92-1ee686f1cadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:34.171358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8d8d4cd9-b50d-467d-bc92-1ee686f1cadc None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:34.172766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8d8d4cd9-b50d-467d-bc92-1ee686f1cadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:34.182380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1021/2039] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:34.186118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44aab9-e29b-484e-b1af-87ccdc1fe1b6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:34.197608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44aab9-e29b-484e-b1af-87ccdc1fe1b6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:34.202708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69b1e2-638f-4e70-8009-0118f3036c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:34.203425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69b1e2-638f-4e70-8009-0118f3036c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:34.203783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69b1e2-638f-4e70-8009-0118f3036c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:34.204062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69b1e2-638f-4e70-8009-0118f3036c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:34.204771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69b1e2-638f-4e70-8009-0118f3036c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:34.205078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69b1e2-638f-4e70-8009-0118f3036c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:34.206720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69b1e2-638f-4e70-8009-0118f3036c39 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:34.220693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1019/2040] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:59:34.225976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d4cd9-b50d-467d-bc92-1ee686f1cadc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:34.236093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d4cd9-b50d-467d-bc92-1ee686f1cadc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:34.243411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837307d6-2a12-4996-bb0f-890f77f637c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:34.244057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837307d6-2a12-4996-bb0f-890f77f637c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:34.244348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837307d6-2a12-4996-bb0f-890f77f637c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:34.244620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837307d6-2a12-4996-bb0f-890f77f637c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:34.245308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837307d6-2a12-4996-bb0f-890f77f637c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:34.247367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837307d6-2a12-4996-bb0f-890f77f637c2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:34.282333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1022/2041] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:34 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:34.287174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69b1e2-638f-4e70-8009-0118f3036c39 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:34.301067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69b1e2-638f-4e70-8009-0118f3036c39 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:34.393822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e6433-8c47-4d0b-a4c6-ab7f9d75598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:34.394468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e6433-8c47-4d0b-a4c6-ab7f9d75598c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:34.395613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e6433-8c47-4d0b-a4c6-ab7f9d75598c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:34.396249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e6433-8c47-4d0b-a4c6-ab7f9d75598c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:34.396925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e6433-8c47-4d0b-a4c6-ab7f9d75598c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:34.399136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e6433-8c47-4d0b-a4c6-ab7f9d75598c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:34.431440 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1020/2042] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:34 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:34.433091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837307d6-2a12-4996-bb0f-890f77f637c2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:34.452530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837307d6-2a12-4996-bb0f-890f77f637c2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:34.458891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b353c5-e666-41ba-a53d-43b3c72c5992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:34.459835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b353c5-e666-41ba-a53d-43b3c72c5992 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:34.460199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b353c5-e666-41ba-a53d-43b3c72c5992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:34.460552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b353c5-e666-41ba-a53d-43b3c72c5992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:34.462870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b353c5-e666-41ba-a53d-43b3c72c5992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:34.464591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b353c5-e666-41ba-a53d-43b3c72c5992 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:34.466209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:34.482666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1023/2043] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:34.483932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e6433-8c47-4d0b-a4c6-ab7f9d75598c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:34.496653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e6433-8c47-4d0b-a4c6-ab7f9d75598c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:34.504064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e298b-d3bf-45f7-80d5-ab0066d05b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:34.505013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e298b-d3bf-45f7-80d5-ab0066d05b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:34.505335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e298b-d3bf-45f7-80d5-ab0066d05b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:34.505770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e298b-d3bf-45f7-80d5-ab0066d05b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:34.507939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e298b-d3bf-45f7-80d5-ab0066d05b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:34.509760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e298b-d3bf-45f7-80d5-ab0066d05b4e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:34.524889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1021/2044] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:34.528743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b353c5-e666-41ba-a53d-43b3c72c5992 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:34.548551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b353c5-e666-41ba-a53d-43b3c72c5992 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:34.561294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd5c65-38f8-4357-bcab-a704b385be22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:34.562008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd5c65-38f8-4357-bcab-a704b385be22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:34.562294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd5c65-38f8-4357-bcab-a704b385be22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:34.562597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd5c65-38f8-4357-bcab-a704b385be22 admin admin] PATH_INFO: `/v3/projects/6d00250cfb0f45e6bbb74e56422d2db2/users/711a2ec788b341eca224a84c288406b1/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:34.616697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd5c65-38f8-4357-bcab-a704b385be22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d00250cfb0f45e6bbb74e56422d2db2, user_id=711a2ec788b341eca224a84c288406b1, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:34.619912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd5c65-38f8-4357-bcab-a704b385be22 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:34.665787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1024/2045] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:34 2022] PUT /identity/v3/projects/6d00250cfb0f45e6bbb74e56422d2db2/users/711a2ec788b341eca224a84c288406b1/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:34.672151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7032e3-383c-4eab-aa0a-439d41446f80 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:34.672660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7032e3-383c-4eab-aa0a-439d41446f80 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:34.672660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7032e3-383c-4eab-aa0a-439d41446f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:34.707054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1e7032e3-383c-4eab-aa0a-439d41446f80 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:34.708948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e7032e3-383c-4eab-aa0a-439d41446f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:34.816724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1022/2046] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:34.820243 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd5c65-38f8-4357-bcab-a704b385be22 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:34.822981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-85fd5c65-38f8-4357-bcab-a704b385be22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:34.989822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd5c65-38f8-4357-bcab-a704b385be22 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:34.999827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b3c24-79f6-4c34-82d1-9a67d0afaf84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:35.000789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b3c24-79f6-4c34-82d1-9a67d0afaf84 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:35.001248 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b3c24-79f6-4c34-82d1-9a67d0afaf84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:35.001598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b3c24-79f6-4c34-82d1-9a67d0afaf84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:35.055706 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b3c24-79f6-4c34-82d1-9a67d0afaf84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:35.057924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b3c24-79f6-4c34-82d1-9a67d0afaf84 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:35.073380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1025/2047] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:36.568675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcf76e-1d8c-44a9-9936-2048ff804f6d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:36.569192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcf76e-1d8c-44a9-9936-2048ff804f6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:36.569283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fcf76e-1d8c-44a9-9936-2048ff804f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:36.595296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:36.630078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-96fcf76e-1d8c-44a9-9936-2048ff804f6d None None] MFA Rules not processed for user `711a2ec788b341eca224a84c288406b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:36.636984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-96fcf76e-1d8c-44a9-9936-2048ff804f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:36.944430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1023/2048] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:36.950905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b3c24-79f6-4c34-82d1-9a67d0afaf84 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:36.982646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b3c24-79f6-4c34-82d1-9a67d0afaf84 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:37.013040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45531723-ff68-42f6-8376-578e9a17d00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:37.017797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45531723-ff68-42f6-8376-578e9a17d00f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:37.019926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45531723-ff68-42f6-8376-578e9a17d00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:37.020612 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45531723-ff68-42f6-8376-578e9a17d00f admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:37.022280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45531723-ff68-42f6-8376-578e9a17d00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:37.030477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45531723-ff68-42f6-8376-578e9a17d00f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:37.114270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1026/2049] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:36 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:37.115599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcf76e-1d8c-44a9-9936-2048ff804f6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:37.132681 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fcf76e-1d8c-44a9-9936-2048ff804f6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:37.354332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a28c6e-0311-47e6-ba53-08251c6e5f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:37.356207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a28c6e-0311-47e6-ba53-08251c6e5f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:37.356499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a28c6e-0311-47e6-ba53-08251c6e5f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:37.356777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a28c6e-0311-47e6-ba53-08251c6e5f01 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:37.360414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a28c6e-0311-47e6-ba53-08251c6e5f01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:37.364106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a28c6e-0311-47e6-ba53-08251c6e5f01 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:37.409328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1024/2050] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:37 2022] POST /identity/v3/users => generated 292 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:37.411676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45531723-ff68-42f6-8376-578e9a17d00f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:37.427493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45531723-ff68-42f6-8376-578e9a17d00f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:37.439049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66957-c3d9-4712-955d-3a878f63a14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:37.442526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66957-c3d9-4712-955d-3a878f63a14d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:37.444042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66957-c3d9-4712-955d-3a878f63a14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:37.444339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc66957-c3d9-4712-955d-3a878f63a14d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:37.446541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc66957-c3d9-4712-955d-3a878f63a14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:37.451568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc66957-c3d9-4712-955d-3a878f63a14d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:37.485980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1027/2051] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:37.490657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a28c6e-0311-47e6-ba53-08251c6e5f01 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:37.515832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a28c6e-0311-47e6-ba53-08251c6e5f01 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:37.532185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bc3fe-20b5-4248-9a2b-916224c6062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:37.533124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bc3fe-20b5-4248-9a2b-916224c6062a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:37.534585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bc3fe-20b5-4248-9a2b-916224c6062a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:37.535067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bc3fe-20b5-4248-9a2b-916224c6062a admin admin] PATH_INFO: `/v3/projects/7ad312edbd634268b08df7b02c842d15/users/f1ccaae8c452440bbe6eb57ec5069c3c/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:37.580309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bc3fe-20b5-4248-9a2b-916224c6062a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad312edbd634268b08df7b02c842d15, user_id=f1ccaae8c452440bbe6eb57ec5069c3c, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:37.583322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bc3fe-20b5-4248-9a2b-916224c6062a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:37.612450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1025/2052] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:37 2022] PUT /identity/v3/projects/7ad312edbd634268b08df7b02c842d15/users/f1ccaae8c452440bbe6eb57ec5069c3c/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:37.614315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66957-c3d9-4712-955d-3a878f63a14d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:37.629241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc66957-c3d9-4712-955d-3a878f63a14d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:37.718039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56097242-3800-40d0-aea5-3f3b117fcc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:37.719225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56097242-3800-40d0-aea5-3f3b117fcc88 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:37.719894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56097242-3800-40d0-aea5-3f3b117fcc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:37.720361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56097242-3800-40d0-aea5-3f3b117fcc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:37.721280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56097242-3800-40d0-aea5-3f3b117fcc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:37.724937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56097242-3800-40d0-aea5-3f3b117fcc88 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:37.743534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1028/2053] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:37.744515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bc3fe-20b5-4248-9a2b-916224c6062a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:37.763991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bc3fe-20b5-4248-9a2b-916224c6062a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:37.771831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9aee2c-d799-4501-b55f-7b18187ebdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:37.772755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9aee2c-d799-4501-b55f-7b18187ebdb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:37.773061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9aee2c-d799-4501-b55f-7b18187ebdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:37.773415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9aee2c-d799-4501-b55f-7b18187ebdb6 admin admin] PATH_INFO: `/v3/projects/7ad312edbd634268b08df7b02c842d15/users/f1ccaae8c452440bbe6eb57ec5069c3c/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:37.776275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9aee2c-d799-4501-b55f-7b18187ebdb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad312edbd634268b08df7b02c842d15, user_id=f1ccaae8c452440bbe6eb57ec5069c3c, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:37.779368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9aee2c-d799-4501-b55f-7b18187ebdb6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:37.799633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1026/2054] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:37 2022] PUT /identity/v3/projects/7ad312edbd634268b08df7b02c842d15/users/f1ccaae8c452440bbe6eb57ec5069c3c/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:38.239021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b0254-a4b8-409b-872c-b0aedac8dd18 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:38.239708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b0254-a4b8-409b-872c-b0aedac8dd18 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:38.240056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b0254-a4b8-409b-872c-b0aedac8dd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:38.271836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6b7b0254-a4b8-409b-872c-b0aedac8dd18 None None] MFA Rules not processed for user `b1a98edf8e5a4a22b3ae7d041b582b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:38.273309 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6b7b0254-a4b8-409b-872c-b0aedac8dd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:38.315832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:38.405732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1029/2055] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:38.412430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9aee2c-d799-4501-b55f-7b18187ebdb6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:38.414208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d9aee2c-d799-4501-b55f-7b18187ebdb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:38.495886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:38.516794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9aee2c-d799-4501-b55f-7b18187ebdb6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:38.522301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf0362-e81e-475d-81a2-4f4d678314c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:38.523162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf0362-e81e-475d-81a2-4f4d678314c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:38.523622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf0362-e81e-475d-81a2-4f4d678314c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:38.524571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf0362-e81e-475d-81a2-4f4d678314c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:38.525330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf0362-e81e-475d-81a2-4f4d678314c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:38.527399 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf0362-e81e-475d-81a2-4f4d678314c4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:38.572380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1027/2056] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:38 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:38.575134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b0254-a4b8-409b-872c-b0aedac8dd18 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:38.607839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b0254-a4b8-409b-872c-b0aedac8dd18 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:38.716006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e8af8-8d17-434d-a92f-904bec9b5bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:38.716600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e8af8-8d17-434d-a92f-904bec9b5bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:38.717045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e8af8-8d17-434d-a92f-904bec9b5bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:38.717442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e8af8-8d17-434d-a92f-904bec9b5bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:38.718244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e8af8-8d17-434d-a92f-904bec9b5bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:38.720578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e8af8-8d17-434d-a92f-904bec9b5bc9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:38.751610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1030/2057] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:38 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:38.756950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf0362-e81e-475d-81a2-4f4d678314c4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:38.773524 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf0362-e81e-475d-81a2-4f4d678314c4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:38.781572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88f86b-2aa7-4e7c-a429-60ed50abcaa5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:38.782107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88f86b-2aa7-4e7c-a429-60ed50abcaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:38.782747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88f86b-2aa7-4e7c-a429-60ed50abcaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:38.783882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0de0f6-e73a-497b-a3e3-16984d63da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:38.784857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de0f6-e73a-497b-a3e3-16984d63da5e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:38.785297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de0f6-e73a-497b-a3e3-16984d63da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:38.786579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de0f6-e73a-497b-a3e3-16984d63da5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:38.788174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0de0f6-e73a-497b-a3e3-16984d63da5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:38.789813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0de0f6-e73a-497b-a3e3-16984d63da5e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:38.823279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1028/2058] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:38.824437 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6c88f86b-2aa7-4e7c-a429-60ed50abcaa5 None None] MFA Rules not processed for user `f1ccaae8c452440bbe6eb57ec5069c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:38.825718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6c88f86b-2aa7-4e7c-a429-60ed50abcaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:38.834961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0de0f6-e73a-497b-a3e3-16984d63da5e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:38.849342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0de0f6-e73a-497b-a3e3-16984d63da5e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:38.856146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a056e26-a499-433a-a11d-ca5ed5fb8946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:38.856839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a056e26-a499-433a-a11d-ca5ed5fb8946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:38.857139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a056e26-a499-433a-a11d-ca5ed5fb8946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:38.857403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a056e26-a499-433a-a11d-ca5ed5fb8946 admin admin] PATH_INFO: `/v3/projects/f96b2dec79d34bc2be795eebfe1b9732/users/73a89bef4f814971aa72a1be13eedf63/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:38.873711 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a056e26-a499-433a-a11d-ca5ed5fb8946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96b2dec79d34bc2be795eebfe1b9732, user_id=73a89bef4f814971aa72a1be13eedf63, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:38.876425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a056e26-a499-433a-a11d-ca5ed5fb8946 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:38.892663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1029/2059] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:38 2022] PUT /identity/v3/projects/f96b2dec79d34bc2be795eebfe1b9732/users/73a89bef4f814971aa72a1be13eedf63/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:38.897881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a056e26-a499-433a-a11d-ca5ed5fb8946 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:38.908936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1031/2060] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:38 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:59:38.923556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88f86b-2aa7-4e7c-a429-60ed50abcaa5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:38.926491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6c88f86b-2aa7-4e7c-a429-60ed50abcaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:38.991409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:39.006178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88f86b-2aa7-4e7c-a429-60ed50abcaa5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:39.006558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a056e26-a499-433a-a11d-ca5ed5fb8946 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:39.012439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9eafc-b5d2-42f0-9135-b6f2371b60f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:39.013227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9eafc-b5d2-42f0-9135-b6f2371b60f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:39.013561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9eafc-b5d2-42f0-9135-b6f2371b60f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:39.013843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9eafc-b5d2-42f0-9135-b6f2371b60f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:39.014574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9eafc-b5d2-42f0-9135-b6f2371b60f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:39.016167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9eafc-b5d2-42f0-9135-b6f2371b60f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:39.038683 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1030/2061] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:39.040191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9eafc-b5d2-42f0-9135-b6f2371b60f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:39.051647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9eafc-b5d2-42f0-9135-b6f2371b60f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:39.056829 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3b080-4153-4367-8ad2-8cbf2796a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:39.057608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3b080-4153-4367-8ad2-8cbf2796a30a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:39.057999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3b080-4153-4367-8ad2-8cbf2796a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:39.058382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3b080-4153-4367-8ad2-8cbf2796a30a admin admin] PATH_INFO: `/v3/projects/f96b2dec79d34bc2be795eebfe1b9732/users/73a89bef4f814971aa72a1be13eedf63/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:39.060628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3b080-4153-4367-8ad2-8cbf2796a30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96b2dec79d34bc2be795eebfe1b9732, user_id=73a89bef4f814971aa72a1be13eedf63, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:39.061937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7906d25-b79d-43cb-9078-1850f62e0ac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:39.062718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7906d25-b79d-43cb-9078-1850f62e0ac0 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:39.063000 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7906d25-b79d-43cb-9078-1850f62e0ac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:39.063272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7906d25-b79d-43cb-9078-1850f62e0ac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:39.063948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3b080-4153-4367-8ad2-8cbf2796a30a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:39.078548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1031/2062] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:39 2022] PUT /identity/v3/projects/f96b2dec79d34bc2be795eebfe1b9732/users/73a89bef4f814971aa72a1be13eedf63/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:39.137760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7906d25-b79d-43cb-9078-1850f62e0ac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:39.139429 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7906d25-b79d-43cb-9078-1850f62e0ac0 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:39.151795 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1032/2063] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:59:43.804013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3b080-4153-4367-8ad2-8cbf2796a30a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:43.899963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3b080-4153-4367-8ad2-8cbf2796a30a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:43.911494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e6ebc-76e1-4cf0-a141-1c032f4cdf37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:43.912577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e6ebc-76e1-4cf0-a141-1c032f4cdf37 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:43.912809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e6ebc-76e1-4cf0-a141-1c032f4cdf37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:43.913129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e6ebc-76e1-4cf0-a141-1c032f4cdf37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:43.990111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:44.027386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e6ebc-76e1-4cf0-a141-1c032f4cdf37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:44.028982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e6ebc-76e1-4cf0-a141-1c032f4cdf37 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:44.042380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1032/2064] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:44.772927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8876e-cbaa-454c-9340-19bc99b08685 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:44.773379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8876e-cbaa-454c-9340-19bc99b08685 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:44.773379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8876e-cbaa-454c-9340-19bc99b08685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:44.812145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bfa8876e-cbaa-454c-9340-19bc99b08685 None None] MFA Rules not processed for user `73a89bef4f814971aa72a1be13eedf63`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:44.813559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bfa8876e-cbaa-454c-9340-19bc99b08685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:44.915059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1033/2065] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:44.929535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e6ebc-76e1-4cf0-a141-1c032f4cdf37 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:44.931887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e90e6ebc-76e1-4cf0-a141-1c032f4cdf37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:45.164811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e6ebc-76e1-4cf0-a141-1c032f4cdf37 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:45.185486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339aa3e-75b3-4023-841a-a94d814d64c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:45.186062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339aa3e-75b3-4023-841a-a94d814d64c0 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:45.186349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339aa3e-75b3-4023-841a-a94d814d64c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:45.186673 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339aa3e-75b3-4023-841a-a94d814d64c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:45.273394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339aa3e-75b3-4023-841a-a94d814d64c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:45.277143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339aa3e-75b3-4023-841a-a94d814d64c0 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:45.292278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1033/2066] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:46.717050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31062c8-24b5-4ad4-ba03-6ab4ba7191b0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:46.718489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31062c8-24b5-4ad4-ba03-6ab4ba7191b0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:46.719117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31062c8-24b5-4ad4-ba03-6ab4ba7191b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:46.769916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c31062c8-24b5-4ad4-ba03-6ab4ba7191b0 None None] MFA Rules not processed for user `b1a98edf8e5a4a22b3ae7d041b582b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:46.771696 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c31062c8-24b5-4ad4-ba03-6ab4ba7191b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:46.789360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1034/2067] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:48.587275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339aa3e-75b3-4023-841a-a94d814d64c0 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:48.589002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c339aa3e-75b3-4023-841a-a94d814d64c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:48.746870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339aa3e-75b3-4023-841a-a94d814d64c0 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:48.755277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:48.755945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:48.756230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:48.756509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] PATH_INFO: `/v3/users/52099395f92347bcabf3c0916d4f283e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:48.758189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52099395f92347bcabf3c0916d4f283e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:48.760523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:48.789648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:48.860518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52099395f92347bcabf3c0916d4f283e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:48.870131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52099395f92347bcabf3c0916d4f283e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:48.871330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52099395f92347bcabf3c0916d4f283e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:48.872153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52099395f92347bcabf3c0916d4f283e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:48.903217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52099395f92347bcabf3c0916d4f283e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:48.940620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1034/2068] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:48 2022] DELETE /identity/v3/users/52099395f92347bcabf3c0916d4f283e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:48.947377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3890e0-ed76-426b-87d0-ea001b19119c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:48.949906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3890e0-ed76-426b-87d0-ea001b19119c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:48.950425 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3890e0-ed76-426b-87d0-ea001b19119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:49.004608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7c3890e0-ed76-426b-87d0-ea001b19119c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:49.006357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7c3890e0-ed76-426b-87d0-ea001b19119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:49.065612 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:49.152374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1035/2069] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:49.180162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:49.181884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:49.273671 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca41d80-94cc-43a6-b868-f171ba015282 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:49.278300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d7ac8-4c87-42e3-8df4-2e88aa182d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:49.279058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d7ac8-4c87-42e3-8df4-2e88aa182d65 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:49.279539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d7ac8-4c87-42e3-8df4-2e88aa182d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:49.279955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d7ac8-4c87-42e3-8df4-2e88aa182d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:49.292105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d7ac8-4c87-42e3-8df4-2e88aa182d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:49.293953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d7ac8-4c87-42e3-8df4-2e88aa182d65 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:49.305233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1035/2070] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:49.620527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3890e0-ed76-426b-87d0-ea001b19119c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:49.622090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7c3890e0-ed76-426b-87d0-ea001b19119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:49.638326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3890e0-ed76-426b-87d0-ea001b19119c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:49.644919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:49.645981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:49.646307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:49.646805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] PATH_INFO: `/v3/projects/b7ca37f9b98d4ae4a4edae28c1f8d13d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:49.648020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ca37f9b98d4ae4a4edae28c1f8d13d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:49.649858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:49.699671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ca37f9b98d4ae4a4edae28c1f8d13d', 'request_id': 'req-c8a14ed2-0326-40b8-a6b9-2e290dd91613'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.706406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ca37f9b98d4ae4a4edae28c1f8d13d', 'request_id': 'req-c8a14ed2-0326-40b8-a6b9-2e290dd91613'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.713172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ca37f9b98d4ae4a4edae28c1f8d13d', 'request_id': 'req-c8a14ed2-0326-40b8-a6b9-2e290dd91613'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.724990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] The token cache is being invalidate because project b7ca37f9b98d4ae4a4edae28c1f8d13d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:49.725279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.725929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.727495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1036/2071] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:49 2022] DELETE /identity/v3/projects/b7ca37f9b98d4ae4a4edae28c1f8d13d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:49.732016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d7ac8-4c87-42e3-8df4-2e88aa182d65 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:49.745533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d7ac8-4c87-42e3-8df4-2e88aa182d65 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:49.809018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:49.809888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:49.810326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:49.810799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] PATH_INFO: `/v3/users/3bdc66ed1d9b48368dba96967e914894` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:49.812252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bdc66ed1d9b48368dba96967e914894)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:49.814587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:49.823489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:49.824794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:49.845046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a14ed2-0326-40b8-a6b9-2e290dd91613 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:49.855800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:49.857719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:49.857995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:49.858316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] PATH_INFO: `/v3/users/a49b6f59d74240e78ea8fa180979c599` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:49.859660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49b6f59d74240e78ea8fa180979c599)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:49.868099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:49.868318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:49.892986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bdc66ed1d9b48368dba96967e914894'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.907246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bdc66ed1d9b48368dba96967e914894'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.908311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bdc66ed1d9b48368dba96967e914894'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.909037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bdc66ed1d9b48368dba96967e914894'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.933364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-24902c3d-4970-4bee-8545-354e36a8d6ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bdc66ed1d9b48368dba96967e914894'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.954070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1036/2072] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:49 2022] DELETE /identity/v3/users/3bdc66ed1d9b48368dba96967e914894 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:49.954939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49b6f59d74240e78ea8fa180979c599'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.965496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49b6f59d74240e78ea8fa180979c599'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.966278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49b6f59d74240e78ea8fa180979c599'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.966967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49b6f59d74240e78ea8fa180979c599'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:49.981656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49b6f59d74240e78ea8fa180979c599'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:50.021009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1037/2073] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:49 2022] DELETE /identity/v3/users/a49b6f59d74240e78ea8fa180979c599 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:50.027204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d52a08-69e4-423c-93ae-ee01f05e805d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:50.027688 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d52a08-69e4-423c-93ae-ee01f05e805d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:50.027969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d52a08-69e4-423c-93ae-ee01f05e805d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:50.056039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-14d52a08-69e4-423c-93ae-ee01f05e805d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:50.057349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-14d52a08-69e4-423c-93ae-ee01f05e805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:50.156278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1037/2074] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:50.175042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:50.177594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:50.222132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:50.310720 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64beee-a456-4767-b9d6-82c98a9a4fc7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:50.319590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6233af-e145-4c20-93e7-ce18d0641f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:50.320223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6233af-e145-4c20-93e7-ce18d0641f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:50.320510 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6233af-e145-4c20-93e7-ce18d0641f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:50.320868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6233af-e145-4c20-93e7-ce18d0641f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:50.334176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d52a08-69e4-423c-93ae-ee01f05e805d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:50.335828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-14d52a08-69e4-423c-93ae-ee01f05e805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:50.350808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6233af-e145-4c20-93e7-ce18d0641f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:50.352353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6233af-e145-4c20-93e7-ce18d0641f03 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:50.369841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d52a08-69e4-423c-93ae-ee01f05e805d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:50.372016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1038/2075] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:50.389002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:50.391452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:50.395108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:50.395767 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] PATH_INFO: `/v3/projects/d992033017e04e79b2e43ad210e881f9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:50.405762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d992033017e04e79b2e43ad210e881f9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:50.413730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:50.543622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd992033017e04e79b2e43ad210e881f9', 'request_id': 'req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:50.555082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd992033017e04e79b2e43ad210e881f9', 'request_id': 'req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:50.585725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd992033017e04e79b2e43ad210e881f9', 'request_id': 'req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:50.613900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] The token cache is being invalidate because project d992033017e04e79b2e43ad210e881f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:50.624518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:50.627326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0726c7e3-6999-45d7-b61d-ed0a3b66dcdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:50.634611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1038/2076] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:50 2022] DELETE /identity/v3/projects/d992033017e04e79b2e43ad210e881f9 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:50.640629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a02ab-b8b2-4bd1-9e51-d68c7d03d110 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:50.641105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a02ab-b8b2-4bd1-9e51-d68c7d03d110 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:50.641588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a02ab-b8b2-4bd1-9e51-d68c7d03d110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:50.735658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8c9a02ab-b8b2-4bd1-9e51-d68c7d03d110 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:50.737178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8c9a02ab-b8b2-4bd1-9e51-d68c7d03d110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:50.840182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1039/2077] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:50.842312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ed96-e108-427c-95b6-4a1772dbc32c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:50.843108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ed96-e108-427c-95b6-4a1772dbc32c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:50.843108 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90ed96-e108-427c-95b6-4a1772dbc32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:50.868136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1b90ed96-e108-427c-95b6-4a1772dbc32c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:50.869460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1b90ed96-e108-427c-95b6-4a1772dbc32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:50.877884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1039/2078] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:50.882935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a02ab-b8b2-4bd1-9e51-d68c7d03d110 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:50.893320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a02ab-b8b2-4bd1-9e51-d68c7d03d110 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:50.898945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e96455-de91-4a9f-a515-f4f3ace8589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:50.899652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e96455-de91-4a9f-a515-f4f3ace8589a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:50.900003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e96455-de91-4a9f-a515-f4f3ace8589a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:50.900346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e96455-de91-4a9f-a515-f4f3ace8589a admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:50.901118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e96455-de91-4a9f-a515-f4f3ace8589a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:50.901513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e96455-de91-4a9f-a515-f4f3ace8589a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:50.903239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e96455-de91-4a9f-a515-f4f3ace8589a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:50.919127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1040/2079] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:59:50.924071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ed96-e108-427c-95b6-4a1772dbc32c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:50.936022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90ed96-e108-427c-95b6-4a1772dbc32c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:50.941943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f37884-c1a1-4d3d-9364-8ec797fa76f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:50.942688 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f37884-c1a1-4d3d-9364-8ec797fa76f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:50.943040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f37884-c1a1-4d3d-9364-8ec797fa76f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:50.943407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f37884-c1a1-4d3d-9364-8ec797fa76f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:50.944152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f37884-c1a1-4d3d-9364-8ec797fa76f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:50.946195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f37884-c1a1-4d3d-9364-8ec797fa76f7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:50.986250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e96455-de91-4a9f-a515-f4f3ace8589a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:50.986833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1040/2080] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:50 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:51.004554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e96455-de91-4a9f-a515-f4f3ace8589a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.086966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980ecfd-ff54-40e2-93ab-6d4a0c73f581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.087730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980ecfd-ff54-40e2-93ab-6d4a0c73f581 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.088105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980ecfd-ff54-40e2-93ab-6d4a0c73f581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.088282 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980ecfd-ff54-40e2-93ab-6d4a0c73f581 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.089079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980ecfd-ff54-40e2-93ab-6d4a0c73f581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.091383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980ecfd-ff54-40e2-93ab-6d4a0c73f581 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.110186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f37884-c1a1-4d3d-9364-8ec797fa76f7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.114982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9f37884-c1a1-4d3d-9364-8ec797fa76f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:51.128101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:51.132334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f37884-c1a1-4d3d-9364-8ec797fa76f7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.138951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.139766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.140250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.140702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] PATH_INFO: `/v3/projects/34670e3e0bb8474a84502e7710cbdc2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.142528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34670e3e0bb8474a84502e7710cbdc2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.143710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1041/2081] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:50 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:51.145531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.147802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980ecfd-ff54-40e2-93ab-6d4a0c73f581 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.183945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980ecfd-ff54-40e2-93ab-6d4a0c73f581 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.207404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c3884-c140-4212-85ce-3ed15d366f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.209808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c3884-c140-4212-85ce-3ed15d366f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.211280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c3884-c140-4212-85ce-3ed15d366f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.211788 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c3884-c140-4212-85ce-3ed15d366f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.212877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c3884-c140-4212-85ce-3ed15d366f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.215030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48c3884-c140-4212-85ce-3ed15d366f00 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.216164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34670e3e0bb8474a84502e7710cbdc2c', 'request_id': 'req-01424d45-a8d1-4514-ad0c-0880b6e62027'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:51.228065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34670e3e0bb8474a84502e7710cbdc2c', 'request_id': 'req-01424d45-a8d1-4514-ad0c-0880b6e62027'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:51.234790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34670e3e0bb8474a84502e7710cbdc2c', 'request_id': 'req-01424d45-a8d1-4514-ad0c-0880b6e62027'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:51.235919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1042/2082] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:51.240258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c3884-c140-4212-85ce-3ed15d366f00 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.250611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] The token cache is being invalidate because project 34670e3e0bb8474a84502e7710cbdc2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:51.250930 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:51.251686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-01424d45-a8d1-4514-ad0c-0880b6e62027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:51.254398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1041/2083] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:51 2022] DELETE /identity/v3/projects/34670e3e0bb8474a84502e7710cbdc2c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:51.255971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c3884-c140-4212-85ce-3ed15d366f00 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.258740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9708eb-bd7a-4dcd-aa3e-ff86546cfe51 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.259307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9708eb-bd7a-4dcd-aa3e-ff86546cfe51 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.259786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9708eb-bd7a-4dcd-aa3e-ff86546cfe51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.285985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5c9708eb-bd7a-4dcd-aa3e-ff86546cfe51 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:51.287862 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5c9708eb-bd7a-4dcd-aa3e-ff86546cfe51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:51.315676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:51.347627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6898d7f-8ba6-425a-a52b-4eb42d8e197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.348766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6898d7f-8ba6-425a-a52b-4eb42d8e197c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.349122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6898d7f-8ba6-425a-a52b-4eb42d8e197c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.349404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6898d7f-8ba6-425a-a52b-4eb42d8e197c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.350900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6898d7f-8ba6-425a-a52b-4eb42d8e197c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.353155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6898d7f-8ba6-425a-a52b-4eb42d8e197c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.367966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1043/2084] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:51.371828 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6898d7f-8ba6-425a-a52b-4eb42d8e197c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.373832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c6898d7f-8ba6-425a-a52b-4eb42d8e197c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:51.386015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1042/2085] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:51.388449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d660d2-a0d9-44b4-91b9-5416c9342c9c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.388744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d660d2-a0d9-44b4-91b9-5416c9342c9c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.389123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d660d2-a0d9-44b4-91b9-5416c9342c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.393453 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6898d7f-8ba6-425a-a52b-4eb42d8e197c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.399166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29a2c4-1381-4c05-af0e-9e31bed6a1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.399987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a2c4-1381-4c05-af0e-9e31bed6a1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.400369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a2c4-1381-4c05-af0e-9e31bed6a1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.400745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29a2c4-1381-4c05-af0e-9e31bed6a1dd admin admin] PATH_INFO: `/v3/projects/1bca3e76a8194552a8e45df7f50312b7/users/3bd2bc168daa4a49ba1e092042e72a81/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.415717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-78d660d2-a0d9-44b4-91b9-5416c9342c9c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:51.417938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-78d660d2-a0d9-44b4-91b9-5416c9342c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:51.423241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a2c4-1381-4c05-af0e-9e31bed6a1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bca3e76a8194552a8e45df7f50312b7, user_id=3bd2bc168daa4a49ba1e092042e72a81, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.426255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29a2c4-1381-4c05-af0e-9e31bed6a1dd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.438877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1043/2086] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:51.445993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d660d2-a0d9-44b4-91b9-5416c9342c9c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.461638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d660d2-a0d9-44b4-91b9-5416c9342c9c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.473783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1044/2087] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:51 2022] PUT /identity/v3/projects/1bca3e76a8194552a8e45df7f50312b7/users/3bd2bc168daa4a49ba1e092042e72a81/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:51.485384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6d43a-3c72-463d-9180-3116a2f4e27b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.485601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6d43a-3c72-463d-9180-3116a2f4e27b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.485870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6d43a-3c72-463d-9180-3116a2f4e27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.517737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7de6d43a-3c72-463d-9180-3116a2f4e27b None None] MFA Rules not processed for user `3bd2bc168daa4a49ba1e092042e72a81`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:51.520182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7de6d43a-3c72-463d-9180-3116a2f4e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:51.536697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f2fc3-b083-41ec-ad71-f285b2f3aa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.537660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f2fc3-b083-41ec-ad71-f285b2f3aa96 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.538841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f2fc3-b083-41ec-ad71-f285b2f3aa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.539323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f2fc3-b083-41ec-ad71-f285b2f3aa96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.540320 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f2fc3-b083-41ec-ad71-f285b2f3aa96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:51.540813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f2fc3-b083-41ec-ad71-f285b2f3aa96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.542676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f2fc3-b083-41ec-ad71-f285b2f3aa96 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.558931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1044/2088] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:59:51.562870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f2fc3-b083-41ec-ad71-f285b2f3aa96 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.573300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f2fc3-b083-41ec-ad71-f285b2f3aa96 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.579535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5107b-170c-4a2a-b5fe-8e50d794c78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.582256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5107b-170c-4a2a-b5fe-8e50d794c78a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.582790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5107b-170c-4a2a-b5fe-8e50d794c78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.583232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5107b-170c-4a2a-b5fe-8e50d794c78a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.584269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5107b-170c-4a2a-b5fe-8e50d794c78a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.586633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5107b-170c-4a2a-b5fe-8e50d794c78a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.607613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1045/2089] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:51.610332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6d43a-3c72-463d-9180-3116a2f4e27b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.613801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:51.622334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6d43a-3c72-463d-9180-3116a2f4e27b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.627375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f44f-63e1-4069-978a-3ed73817a33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.627999 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f44f-63e1-4069-978a-3ed73817a33a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.628319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f44f-63e1-4069-978a-3ed73817a33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.628605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f44f-63e1-4069-978a-3ed73817a33a admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.629507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51f44f-63e1-4069-978a-3ed73817a33a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.631618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51f44f-63e1-4069-978a-3ed73817a33a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.637349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1045/2090] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:51 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:51.638345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5107b-170c-4a2a-b5fe-8e50d794c78a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.670135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1046/2091] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:51 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:51.671410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f44f-63e1-4069-978a-3ed73817a33a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.725839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5107b-170c-4a2a-b5fe-8e50d794c78a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.766790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f44f-63e1-4069-978a-3ed73817a33a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.777377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d271a3-94f5-4292-83c4-fee81108b87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.778223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d271a3-94f5-4292-83c4-fee81108b87e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.778470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d271a3-94f5-4292-83c4-fee81108b87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.780043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d271a3-94f5-4292-83c4-fee81108b87e admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.781177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d271a3-94f5-4292-83c4-fee81108b87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.787388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d271a3-94f5-4292-83c4-fee81108b87e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.857072 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1047/2092] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:51 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:51.858066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d271a3-94f5-4292-83c4-fee81108b87e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.873409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:51.882044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d271a3-94f5-4292-83c4-fee81108b87e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.899926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0979c5e-6de9-4163-9ebc-5eada5695ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.900722 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979c5e-6de9-4163-9ebc-5eada5695ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.901034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979c5e-6de9-4163-9ebc-5eada5695ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.901244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979c5e-6de9-4163-9ebc-5eada5695ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.901673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea41b0f-26c6-429e-a03e-0ca6f1f26fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.901922 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0979c5e-6de9-4163-9ebc-5eada5695ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.902323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea41b0f-26c6-429e-a03e-0ca6f1f26fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.902655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea41b0f-26c6-429e-a03e-0ca6f1f26fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.902926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea41b0f-26c6-429e-a03e-0ca6f1f26fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.903631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea41b0f-26c6-429e-a03e-0ca6f1f26fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.903966 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0979c5e-6de9-4163-9ebc-5eada5695ab0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.905245 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea41b0f-26c6-429e-a03e-0ca6f1f26fa8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.922427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1048/2093] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:51.928173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea41b0f-26c6-429e-a03e-0ca6f1f26fa8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.945792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1046/2094] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:51 2022] POST /identity/v3/users => generated 288 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:51.949755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0979c5e-6de9-4163-9ebc-5eada5695ab0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:51.959941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea41b0f-26c6-429e-a03e-0ca6f1f26fa8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.963450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0979c5e-6de9-4163-9ebc-5eada5695ab0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:51.965038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9a846-97cb-4e19-9260-3850fb9efbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.965721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9a846-97cb-4e19-9260-3850fb9efbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.965997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9a846-97cb-4e19-9260-3850fb9efbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.966417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9a846-97cb-4e19-9260-3850fb9efbf5 admin admin] PATH_INFO: `/v3/projects/16f50d42a36248e9b470c14b52f84034/users/477c9171f76340d384e1c3fa384a4283/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.974405 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764eb43-b3bc-460e-abe9-cc67f98b0e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:51.983468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764eb43-b3bc-460e-abe9-cc67f98b0e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:51.983950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764eb43-b3bc-460e-abe9-cc67f98b0e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:51.984335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764eb43-b3bc-460e-abe9-cc67f98b0e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:51.985202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764eb43-b3bc-460e-abe9-cc67f98b0e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.987093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764eb43-b3bc-460e-abe9-cc67f98b0e88 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:51.992755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9a846-97cb-4e19-9260-3850fb9efbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f50d42a36248e9b470c14b52f84034, user_id=477c9171f76340d384e1c3fa384a4283, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:51.995646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9a846-97cb-4e19-9260-3850fb9efbf5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:52.007318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1047/2095] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:52.009407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764eb43-b3bc-460e-abe9-cc67f98b0e88 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:52.038601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1049/2096] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:51 2022] PUT /identity/v3/projects/16f50d42a36248e9b470c14b52f84034/users/477c9171f76340d384e1c3fa384a4283/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:52.039192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764eb43-b3bc-460e-abe9-cc67f98b0e88 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:52.046363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9a846-97cb-4e19-9260-3850fb9efbf5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:52.146093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9a846-97cb-4e19-9260-3850fb9efbf5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:52.155651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce514a-63b4-4315-a5aa-b14d8d729c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:52.157204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce514a-63b4-4315-a5aa-b14d8d729c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:52.157496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce514a-63b4-4315-a5aa-b14d8d729c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:52.157771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce514a-63b4-4315-a5aa-b14d8d729c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:52.158509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce514a-63b4-4315-a5aa-b14d8d729c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:52.161189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce514a-63b4-4315-a5aa-b14d8d729c40 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:52.161709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576860e9-5b45-461c-b82b-aba3ebbe2692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:52.162981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576860e9-5b45-461c-b82b-aba3ebbe2692 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:52.164250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576860e9-5b45-461c-b82b-aba3ebbe2692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:52.164745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576860e9-5b45-461c-b82b-aba3ebbe2692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:52.166229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576860e9-5b45-461c-b82b-aba3ebbe2692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:52.170068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576860e9-5b45-461c-b82b-aba3ebbe2692 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:52.176324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1050/2097] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:52.180675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce514a-63b4-4315-a5aa-b14d8d729c40 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:52.193932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce514a-63b4-4315-a5aa-b14d8d729c40 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:52.195648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1048/2098] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:52.198939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576860e9-5b45-461c-b82b-aba3ebbe2692 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:52.225221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe107da-9311-4091-a628-f2c266cecc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:52.225221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe107da-9311-4091-a628-f2c266cecc15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:52.225221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe107da-9311-4091-a628-f2c266cecc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:52.225221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe107da-9311-4091-a628-f2c266cecc15 admin admin] PATH_INFO: `/v3/projects/16f50d42a36248e9b470c14b52f84034/users/477c9171f76340d384e1c3fa384a4283/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:52.225534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe107da-9311-4091-a628-f2c266cecc15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f50d42a36248e9b470c14b52f84034, user_id=477c9171f76340d384e1c3fa384a4283, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:52.240865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe107da-9311-4091-a628-f2c266cecc15 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:52.242858 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576860e9-5b45-461c-b82b-aba3ebbe2692 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:52.248851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8bb1b-5f2f-4429-9e1b-63c4083fe713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:52.249680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8bb1b-5f2f-4429-9e1b-63c4083fe713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:52.250209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8bb1b-5f2f-4429-9e1b-63c4083fe713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:52.250758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8bb1b-5f2f-4429-9e1b-63c4083fe713 admin admin] PATH_INFO: `/v3/projects/5dc01f77fe9745369cef62c8379f9209/users/929d7587f49445ed9c86d53a1d888648/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:52.270798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:52.288893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1051/2099] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:52 2022] PUT /identity/v3/projects/16f50d42a36248e9b470c14b52f84034/users/477c9171f76340d384e1c3fa384a4283/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:52.289143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8bb1b-5f2f-4429-9e1b-63c4083fe713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc01f77fe9745369cef62c8379f9209, user_id=929d7587f49445ed9c86d53a1d888648, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:52.289143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8bb1b-5f2f-4429-9e1b-63c4083fe713 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:52.290844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a30a48-5b40-429b-acb7-8af077ac33ab None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:52.291440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a30a48-5b40-429b-acb7-8af077ac33ab None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:52.291959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a30a48-5b40-429b-acb7-8af077ac33ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:52.340866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1049/2100] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:52 2022] PUT /identity/v3/projects/5dc01f77fe9745369cef62c8379f9209/users/929d7587f49445ed9c86d53a1d888648/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:52.348230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5501f7d-31e4-4bc1-bb08-cc2782fa6647 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:52.348517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5501f7d-31e4-4bc1-bb08-cc2782fa6647 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:52.348794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5501f7d-31e4-4bc1-bb08-cc2782fa6647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:52.356936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-46a30a48-5b40-429b-acb7-8af077ac33ab None None] MFA Rules not processed for user `477c9171f76340d384e1c3fa384a4283`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:52.358405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-46a30a48-5b40-429b-acb7-8af077ac33ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:52.419571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5501f7d-31e4-4bc1-bb08-cc2782fa6647 None None] MFA Rules not processed for user `929d7587f49445ed9c86d53a1d888648`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:52.421074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5501f7d-31e4-4bc1-bb08-cc2782fa6647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:52.478673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1052/2101] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:52.493117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a30a48-5b40-429b-acb7-8af077ac33ab None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:52.494772 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-46a30a48-5b40-429b-acb7-8af077ac33ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:52.531136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1050/2102] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:52.544484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5501f7d-31e4-4bc1-bb08-cc2782fa6647 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:52.546057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5501f7d-31e4-4bc1-bb08-cc2782fa6647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:52.554013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:52.573445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a30a48-5b40-429b-acb7-8af077ac33ab None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:52.586115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea234525-7336-4a35-9e83-9aa854c317c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:52.587754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea234525-7336-4a35-9e83-9aa854c317c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:52.589648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea234525-7336-4a35-9e83-9aa854c317c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:52.589773 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea234525-7336-4a35-9e83-9aa854c317c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:52.604742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea234525-7336-4a35-9e83-9aa854c317c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:52.607179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea234525-7336-4a35-9e83-9aa854c317c6 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:52.625824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5501f7d-31e4-4bc1-bb08-cc2782fa6647 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:52.627777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1053/2103] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:52.636585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc080940-50e8-4257-923f-be156c5db4d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:52.637233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc080940-50e8-4257-923f-be156c5db4d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:52.637881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc080940-50e8-4257-923f-be156c5db4d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:52.638209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc080940-50e8-4257-923f-be156c5db4d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:52.659019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc080940-50e8-4257-923f-be156c5db4d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:52.661516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc080940-50e8-4257-923f-be156c5db4d2 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:52.675900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1051/2104] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:53.299064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea234525-7336-4a35-9e83-9aa854c317c6 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:53.413824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea234525-7336-4a35-9e83-9aa854c317c6 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:53.434091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:53.434943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:53.435561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:53.435951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] PATH_INFO: `/v3/users/3bd2bc168daa4a49ba1e092042e72a81` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:53.438422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd2bc168daa4a49ba1e092042e72a81)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:53.446218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:53.518050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd2bc168daa4a49ba1e092042e72a81'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.526860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd2bc168daa4a49ba1e092042e72a81'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.527662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd2bc168daa4a49ba1e092042e72a81'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.528277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd2bc168daa4a49ba1e092042e72a81'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.545869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd2bc168daa4a49ba1e092042e72a81'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.576632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1054/2105] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:53 2022] DELETE /identity/v3/users/3bd2bc168daa4a49ba1e092042e72a81 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:53.584782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92382ab-4df7-4ec3-9cee-082132004e1e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:53.585275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92382ab-4df7-4ec3-9cee-082132004e1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:53.585373 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92382ab-4df7-4ec3-9cee-082132004e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:53.607835 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e92382ab-4df7-4ec3-9cee-082132004e1e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:53.609185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e92382ab-4df7-4ec3-9cee-082132004e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:53.685024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1052/2106] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:53.702579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:53.704279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:53.715342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92382ab-4df7-4ec3-9cee-082132004e1e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:53.727546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e92382ab-4df7-4ec3-9cee-082132004e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:53.742276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92382ab-4df7-4ec3-9cee-082132004e1e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:53.747598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:53.748244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:53.748531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:53.748809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] PATH_INFO: `/v3/users/691a9c5e4d534dcc9c26f8cf88cd7fb5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:53.750001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=691a9c5e4d534dcc9c26f8cf88cd7fb5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:53.751963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:53.773600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a6612-c321-45e1-8348-eb47c9cfb8b4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:53.775554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:53.777732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f0487-3748-416f-87e9-a8574956e579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:53.777892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f0487-3748-416f-87e9-a8574956e579 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:53.778172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f0487-3748-416f-87e9-a8574956e579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:53.778465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f0487-3748-416f-87e9-a8574956e579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:53.793579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f0487-3748-416f-87e9-a8574956e579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:53.795215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f0487-3748-416f-87e9-a8574956e579 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:53.805501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1055/2107] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:53.809952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691a9c5e4d534dcc9c26f8cf88cd7fb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.817827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691a9c5e4d534dcc9c26f8cf88cd7fb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.818658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691a9c5e4d534dcc9c26f8cf88cd7fb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.819344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691a9c5e4d534dcc9c26f8cf88cd7fb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.835224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691a9c5e4d534dcc9c26f8cf88cd7fb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:53.865926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1053/2108] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:53 2022] DELETE /identity/v3/users/691a9c5e4d534dcc9c26f8cf88cd7fb5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:54.105341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f0487-3748-416f-87e9-a8574956e579 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:54.111206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:54.121001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f0487-3748-416f-87e9-a8574956e579 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:54.160203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:54.161638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:54.215087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:54.215736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.216013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.216336 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] PATH_INFO: `/v3/projects/ef5fe5ac675540c0930bd7bab0d8b27f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.217508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef5fe5ac675540c0930bd7bab0d8b27f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:54.219485 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:54.239682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d4ae3-c0d7-4730-b2c4-6f7c03505c98 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:54.246883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:54.247550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.247943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.248332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] PATH_INFO: `/v3/projects/1bca3e76a8194552a8e45df7f50312b7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.250039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bca3e76a8194552a8e45df7f50312b7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:54.252463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:54.286427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef5fe5ac675540c0930bd7bab0d8b27f', 'request_id': 'req-3b8497d6-4f88-4676-9d54-8101fa1ff74d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.302970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef5fe5ac675540c0930bd7bab0d8b27f', 'request_id': 'req-3b8497d6-4f88-4676-9d54-8101fa1ff74d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.309551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef5fe5ac675540c0930bd7bab0d8b27f', 'request_id': 'req-3b8497d6-4f88-4676-9d54-8101fa1ff74d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.317953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bca3e76a8194552a8e45df7f50312b7', 'request_id': 'req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.323484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] The token cache is being invalidate because project ef5fe5ac675540c0930bd7bab0d8b27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:54.323910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.324836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b8497d6-4f88-4676-9d54-8101fa1ff74d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.326400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bca3e76a8194552a8e45df7f50312b7', 'request_id': 'req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.326672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1056/2109] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:54 2022] DELETE /identity/v3/projects/ef5fe5ac675540c0930bd7bab0d8b27f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:54.327932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261c71d-b099-4818-b3a3-3728394e8163 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.328215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261c71d-b099-4818-b3a3-3728394e8163 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.328486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261c71d-b099-4818-b3a3-3728394e8163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.337284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bca3e76a8194552a8e45df7f50312b7', 'request_id': 'req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.348727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] The token cache is being invalidate because project 1bca3e76a8194552a8e45df7f50312b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:54.349015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.349234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3261c71d-b099-4818-b3a3-3728394e8163 None None] MFA Rules not processed for user `929d7587f49445ed9c86d53a1d888648`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:54.349705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f173fc2b-c8a6-45f2-9e10-aef13b1d9d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.350475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3261c71d-b099-4818-b3a3-3728394e8163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:54.351349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1054/2110] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:54 2022] DELETE /identity/v3/projects/1bca3e76a8194552a8e45df7f50312b7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:54.354427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fba879-48b6-4bce-8fd8-780a9935e60b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.354427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fba879-48b6-4bce-8fd8-780a9935e60b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.354427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fba879-48b6-4bce-8fd8-780a9935e60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.376325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-90fba879-48b6-4bce-8fd8-780a9935e60b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:54.377452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-90fba879-48b6-4bce-8fd8-780a9935e60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:54.401700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:54.446475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1057/2111] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:54.452111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261c71d-b099-4818-b3a3-3728394e8163 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:54.454285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3261c71d-b099-4818-b3a3-3728394e8163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:54.461391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1055/2112] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:54.470171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344db7ae-305d-4e73-8794-be06b0ebd26d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.470554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344db7ae-305d-4e73-8794-be06b0ebd26d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.470892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344db7ae-305d-4e73-8794-be06b0ebd26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.487959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261c71d-b099-4818-b3a3-3728394e8163 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:54.494866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:54.495739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-344db7ae-305d-4e73-8794-be06b0ebd26d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:54.496426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.496719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.496995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] PATH_INFO: `/v3/users/477c9171f76340d384e1c3fa384a4283` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.497280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-344db7ae-305d-4e73-8794-be06b0ebd26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:54.498869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=477c9171f76340d384e1c3fa384a4283)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:54.501625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:54.511106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1056/2113] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:54.513541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344db7ae-305d-4e73-8794-be06b0ebd26d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:54.531601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344db7ae-305d-4e73-8794-be06b0ebd26d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:54.555015 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefaad6-1611-4571-a5c0-f17489bb9425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:54.555860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefaad6-1611-4571-a5c0-f17489bb9425 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.556205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefaad6-1611-4571-a5c0-f17489bb9425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.556492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefaad6-1611-4571-a5c0-f17489bb9425 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.557538 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefaad6-1611-4571-a5c0-f17489bb9425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:54.557985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefaad6-1611-4571-a5c0-f17489bb9425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:54.560083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefaad6-1611-4571-a5c0-f17489bb9425 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:54.564712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:54.577202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1057/2114] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 04:59:54.581531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefaad6-1611-4571-a5c0-f17489bb9425 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:54.584094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '477c9171f76340d384e1c3fa384a4283'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.593393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '477c9171f76340d384e1c3fa384a4283'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.594241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '477c9171f76340d384e1c3fa384a4283'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.595087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '477c9171f76340d384e1c3fa384a4283'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.595462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefaad6-1611-4571-a5c0-f17489bb9425 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:54.600797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8832739-d5f5-4d66-9f73-37eaafda9d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:54.601427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8832739-d5f5-4d66-9f73-37eaafda9d6f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.601833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8832739-d5f5-4d66-9f73-37eaafda9d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.602115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8832739-d5f5-4d66-9f73-37eaafda9d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.603040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8832739-d5f5-4d66-9f73-37eaafda9d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:54.605196 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8832739-d5f5-4d66-9f73-37eaafda9d6f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:54.613034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '477c9171f76340d384e1c3fa384a4283'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:54.635315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1058/2115] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:54 2022] DELETE /identity/v3/users/477c9171f76340d384e1c3fa384a4283 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:54.649168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1058/2116] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:54 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:54.653251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:54.654705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:54.756620 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c97fbf-fd6d-4d5b-a8e9-51cd2af156d1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:54.846945 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2acf9d-0d04-4bc7-b59c-7bbffdcf9f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:54.848094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2acf9d-0d04-4bc7-b59c-7bbffdcf9f78 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.848094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2acf9d-0d04-4bc7-b59c-7bbffdcf9f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.848430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2acf9d-0d04-4bc7-b59c-7bbffdcf9f78 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.849365 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2acf9d-0d04-4bc7-b59c-7bbffdcf9f78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:54.852168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2acf9d-0d04-4bc7-b59c-7bbffdcf9f78 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:54.891301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1059/2117] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:54 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:54.892248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8832739-d5f5-4d66-9f73-37eaafda9d6f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:54.905183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8832739-d5f5-4d66-9f73-37eaafda9d6f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:54.911083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc4e9a-5c79-448f-a0a3-afa879f20fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:54.912423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc4e9a-5c79-448f-a0a3-afa879f20fec admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.912423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc4e9a-5c79-448f-a0a3-afa879f20fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.912423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc4e9a-5c79-448f-a0a3-afa879f20fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.913395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc4e9a-5c79-448f-a0a3-afa879f20fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:54.914716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc4e9a-5c79-448f-a0a3-afa879f20fec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:54.930477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1059/2118] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:54.933281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2acf9d-0d04-4bc7-b59c-7bbffdcf9f78 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:54.950640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2acf9d-0d04-4bc7-b59c-7bbffdcf9f78 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:54.962689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c9388-fc70-4cb1-b9a2-cff89fb715d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:54.963497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c9388-fc70-4cb1-b9a2-cff89fb715d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:54.963787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c9388-fc70-4cb1-b9a2-cff89fb715d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:54.964058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c9388-fc70-4cb1-b9a2-cff89fb715d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:54.964770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c9388-fc70-4cb1-b9a2-cff89fb715d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:54.966400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c9388-fc70-4cb1-b9a2-cff89fb715d1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:54.983405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1060/2119] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:54.984827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc4e9a-5c79-448f-a0a3-afa879f20fec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.001341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc4e9a-5c79-448f-a0a3-afa879f20fec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.015193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32cf5a-b6a9-4c68-b655-d2de98ccf8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.016176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32cf5a-b6a9-4c68-b655-d2de98ccf8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.016522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32cf5a-b6a9-4c68-b655-d2de98ccf8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.023530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32cf5a-b6a9-4c68-b655-d2de98ccf8ab admin admin] PATH_INFO: `/v3/projects/c99e869e8f634c5b86d9939d3d168657/users/4412c814ea3d4ed39281696df34ededa/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.035002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c9388-fc70-4cb1-b9a2-cff89fb715d1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.037154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d17c9388-fc70-4cb1-b9a2-cff89fb715d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:55.058291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32cf5a-b6a9-4c68-b655-d2de98ccf8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99e869e8f634c5b86d9939d3d168657, user_id=4412c814ea3d4ed39281696df34ededa, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.061164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32cf5a-b6a9-4c68-b655-d2de98ccf8ab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.068742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c9388-fc70-4cb1-b9a2-cff89fb715d1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.076544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.077472 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.077862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.078240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] PATH_INFO: `/v3/projects/16f50d42a36248e9b470c14b52f84034` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.079794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f50d42a36248e9b470c14b52f84034)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.082630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.086541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1060/2120] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:54 2022] PUT /identity/v3/projects/c99e869e8f634c5b86d9939d3d168657/users/4412c814ea3d4ed39281696df34ededa/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:55.095004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc53b85-8ec7-4e3b-96b3-907a9f1f7a13 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.102587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc53b85-8ec7-4e3b-96b3-907a9f1f7a13 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.102928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc53b85-8ec7-4e3b-96b3-907a9f1f7a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.155819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-afc53b85-8ec7-4e3b-96b3-907a9f1f7a13 None None] MFA Rules not processed for user `4412c814ea3d4ed39281696df34ededa`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:55.157378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-afc53b85-8ec7-4e3b-96b3-907a9f1f7a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:55.168257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:55.196831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f50d42a36248e9b470c14b52f84034', 'request_id': 'req-a54d2b97-3de4-4617-b1b8-35c9274b5918'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.210611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f50d42a36248e9b470c14b52f84034', 'request_id': 'req-a54d2b97-3de4-4617-b1b8-35c9274b5918'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.223022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f50d42a36248e9b470c14b52f84034', 'request_id': 'req-a54d2b97-3de4-4617-b1b8-35c9274b5918'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.254473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] The token cache is being invalidate because project 16f50d42a36248e9b470c14b52f84034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:55.254777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.258590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a54d2b97-3de4-4617-b1b8-35c9274b5918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.260176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1061/2121] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:55 2022] DELETE /identity/v3/projects/16f50d42a36248e9b470c14b52f84034 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:55.268119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307ef40-1c15-4062-b9e5-e3af8b6c048a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.268505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307ef40-1c15-4062-b9e5-e3af8b6c048a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.269684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307ef40-1c15-4062-b9e5-e3af8b6c048a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.296514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1061/2122] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:59:55.308872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc53b85-8ec7-4e3b-96b3-907a9f1f7a13 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.310403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-afc53b85-8ec7-4e3b-96b3-907a9f1f7a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:55.312176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3307ef40-1c15-4062-b9e5-e3af8b6c048a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:55.313431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3307ef40-1c15-4062-b9e5-e3af8b6c048a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:55.374169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:55.391422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1062/2123] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:55.399609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fb0a9-d183-4265-8640-4cb714d3fa06 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.400887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fb0a9-d183-4265-8640-4cb714d3fa06 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.401242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435fb0a9-d183-4265-8640-4cb714d3fa06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.421110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc53b85-8ec7-4e3b-96b3-907a9f1f7a13 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.425924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-435fb0a9-d183-4265-8640-4cb714d3fa06 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:55.427014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160757b2-4c95-4bd7-aa21-bc3c209fceae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.427456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-435fb0a9-d183-4265-8640-4cb714d3fa06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:55.427768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160757b2-4c95-4bd7-aa21-bc3c209fceae service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.428123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160757b2-4c95-4bd7-aa21-bc3c209fceae service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.428535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160757b2-4c95-4bd7-aa21-bc3c209fceae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.437393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1063/2124] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:55.438578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160757b2-4c95-4bd7-aa21-bc3c209fceae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.440394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160757b2-4c95-4bd7-aa21-bc3c209fceae service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.442163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fb0a9-d183-4265-8640-4cb714d3fa06 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.452272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435fb0a9-d183-4265-8640-4cb714d3fa06 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.452918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1062/2125] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:55.461192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb599915-958b-496c-baba-7dae056961b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.461824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb599915-958b-496c-baba-7dae056961b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.462098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb599915-958b-496c-baba-7dae056961b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.462366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb599915-958b-496c-baba-7dae056961b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.463099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb599915-958b-496c-baba-7dae056961b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:55.463399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb599915-958b-496c-baba-7dae056961b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.464974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb599915-958b-496c-baba-7dae056961b4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.478489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1064/2126] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:59:55.483910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160757b2-4c95-4bd7-aa21-bc3c209fceae service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.500609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160757b2-4c95-4bd7-aa21-bc3c209fceae service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.506142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f7b2c-2133-424b-95ed-90960b5b3cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.506796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f7b2c-2133-424b-95ed-90960b5b3cbe admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.507081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f7b2c-2133-424b-95ed-90960b5b3cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.507378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f7b2c-2133-424b-95ed-90960b5b3cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.508028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f7b2c-2133-424b-95ed-90960b5b3cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.509920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f7b2c-2133-424b-95ed-90960b5b3cbe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.547953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1063/2127] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:55 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:55.552876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb599915-958b-496c-baba-7dae056961b4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.564138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb599915-958b-496c-baba-7dae056961b4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.645578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a6d67-96cc-4894-b9ee-3b5405d87224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.646379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a6d67-96cc-4894-b9ee-3b5405d87224 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.646842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a6d67-96cc-4894-b9ee-3b5405d87224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.647120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a6d67-96cc-4894-b9ee-3b5405d87224 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.649710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a6d67-96cc-4894-b9ee-3b5405d87224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.663571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a6d67-96cc-4894-b9ee-3b5405d87224 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.701306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1065/2128] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:55 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:55.702593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f7b2c-2133-424b-95ed-90960b5b3cbe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.719582 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f7b2c-2133-424b-95ed-90960b5b3cbe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.727061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6897b-703e-453e-8d29-de433579000d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.728080 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6897b-703e-453e-8d29-de433579000d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.728544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6897b-703e-453e-8d29-de433579000d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.728981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f6897b-703e-453e-8d29-de433579000d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.736652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6897b-703e-453e-8d29-de433579000d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.739101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f6897b-703e-453e-8d29-de433579000d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.742555 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a6d67-96cc-4894-b9ee-3b5405d87224 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.744359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0b0a6d67-96cc-4894-b9ee-3b5405d87224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:55.758985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a6d67-96cc-4894-b9ee-3b5405d87224 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.760670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1064/2129] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:55.764882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6897b-703e-453e-8d29-de433579000d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.777221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.777221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.777523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.777906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] PATH_INFO: `/v3/users/4412c814ea3d4ed39281696df34ededa` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.779317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4412c814ea3d4ed39281696df34ededa)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.781304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.785018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f6897b-703e-453e-8d29-de433579000d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.793422 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355d71d-1ea0-4c7d-a5b5-f3ca836d9f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.794185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355d71d-1ea0-4c7d-a5b5-f3ca836d9f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.794497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355d71d-1ea0-4c7d-a5b5-f3ca836d9f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.794896 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355d71d-1ea0-4c7d-a5b5-f3ca836d9f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.796310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355d71d-1ea0-4c7d-a5b5-f3ca836d9f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.798575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355d71d-1ea0-4c7d-a5b5-f3ca836d9f5c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.809765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1065/2130] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:55.814010 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355d71d-1ea0-4c7d-a5b5-f3ca836d9f5c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:55.814929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:55.825643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355d71d-1ea0-4c7d-a5b5-f3ca836d9f5c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:55.832201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe4760-879b-4839-b2dd-e549f68b09a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:55.833092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe4760-879b-4839-b2dd-e549f68b09a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.833547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe4760-879b-4839-b2dd-e549f68b09a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.834037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe4760-879b-4839-b2dd-e549f68b09a6 admin admin] PATH_INFO: `/v3/projects/b814b33e4f2f4a84ab57e7588bc0714e/users/093c939da58448468abdda19067ad523/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.848467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4412c814ea3d4ed39281696df34ededa'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.857833 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4412c814ea3d4ed39281696df34ededa'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.858188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe4760-879b-4839-b2dd-e549f68b09a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b814b33e4f2f4a84ab57e7588bc0714e, user_id=093c939da58448468abdda19067ad523, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:55.858748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4412c814ea3d4ed39281696df34ededa'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.859395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4412c814ea3d4ed39281696df34ededa'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.862256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe4760-879b-4839-b2dd-e549f68b09a6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:55.880608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1066/2131] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:55 2022] PUT /identity/v3/projects/b814b33e4f2f4a84ab57e7588bc0714e/users/093c939da58448468abdda19067ad523/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:55.890678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f482-8b20-44b5-aa13-63fdd981c0c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.891183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f482-8b20-44b5-aa13-63fdd981c0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.891463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4f482-8b20-44b5-aa13-63fdd981c0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.899806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d6a94609-49c8-46ea-916e-f76b7acf4076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4412c814ea3d4ed39281696df34ededa'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:55.926877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1066/2132] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:55 2022] DELETE /identity/v3/users/4412c814ea3d4ed39281696df34ededa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:55.927375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-10f4f482-8b20-44b5-aa13-63fdd981c0c8 None None] MFA Rules not processed for user `093c939da58448468abdda19067ad523`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:55.928933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10f4f482-8b20-44b5-aa13-63fdd981c0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:55.931267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9429d1-709a-4020-93fe-ef669bfd99c2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:55.931581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9429d1-709a-4020-93fe-ef669bfd99c2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:55.931888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9429d1-709a-4020-93fe-ef669bfd99c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:55.953263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bf9429d1-709a-4020-93fe-ef669bfd99c2 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:55.954515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bf9429d1-709a-4020-93fe-ef669bfd99c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:56.000575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1067/2133] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:59:56.012337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4f482-8b20-44b5-aa13-63fdd981c0c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:56.013911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10f4f482-8b20-44b5-aa13-63fdd981c0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:56.014723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1067/2134] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 04:59:56.026115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9429d1-709a-4020-93fe-ef669bfd99c2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:56.027329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bf9429d1-709a-4020-93fe-ef669bfd99c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:56.072536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:56.078913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4f482-8b20-44b5-aa13-63fdd981c0c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:56.091745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d322d3-2de3-4e65-b986-e4dcbcd91171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:56.093059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d322d3-2de3-4e65-b986-e4dcbcd91171 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:56.093285 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9429d1-709a-4020-93fe-ef669bfd99c2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:56.097723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb0cf0-53b9-4fe6-97f4-d51619cd05f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:56.098369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb0cf0-53b9-4fe6-97f4-d51619cd05f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:56.098684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb0cf0-53b9-4fe6-97f4-d51619cd05f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:56.099100 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb0cf0-53b9-4fe6-97f4-d51619cd05f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:56.100207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d322d3-2de3-4e65-b986-e4dcbcd91171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:56.100883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d322d3-2de3-4e65-b986-e4dcbcd91171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:56.110255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb0cf0-53b9-4fe6-97f4-d51619cd05f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:56.110849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d322d3-2de3-4e65-b986-e4dcbcd91171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:56.111725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb0cf0-53b9-4fe6-97f4-d51619cd05f0 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:56.113619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d322d3-2de3-4e65-b986-e4dcbcd91171 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:56.122427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1068/2135] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:59:56.128678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1068/2136] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 04:59:56.708383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb0cf0-53b9-4fe6-97f4-d51619cd05f0 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:56.709990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-70cb0cf0-53b9-4fe6-97f4-d51619cd05f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:56.732815 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb0cf0-53b9-4fe6-97f4-d51619cd05f0 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:56.747435 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:56.748278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:56.748624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:56.750081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] PATH_INFO: `/v3/projects/c99e869e8f634c5b86d9939d3d168657` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:56.751522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99e869e8f634c5b86d9939d3d168657)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:56.755242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:56.825951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:56.842803 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99e869e8f634c5b86d9939d3d168657', 'request_id': 'req-090d9b85-ead3-4731-9595-baad3a6a21ca'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:56.853521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99e869e8f634c5b86d9939d3d168657', 'request_id': 'req-090d9b85-ead3-4731-9595-baad3a6a21ca'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:56.861175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99e869e8f634c5b86d9939d3d168657', 'request_id': 'req-090d9b85-ead3-4731-9595-baad3a6a21ca'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:56.876404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] The token cache is being invalidate because project c99e869e8f634c5b86d9939d3d168657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 04:59:56.876833 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:56.877784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-090d9b85-ead3-4731-9595-baad3a6a21ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:56.883563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1069/2137] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 04:59:56 2022] DELETE /identity/v3/projects/c99e869e8f634c5b86d9939d3d168657 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:56.885359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c93b8-3315-4819-9537-8622a6ca1fef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:56.885831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c93b8-3315-4819-9537-8622a6ca1fef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:56.886188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c93b8-3315-4819-9537-8622a6ca1fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:56.907947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-634c93b8-3315-4819-9537-8622a6ca1fef None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:56.910287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-634c93b8-3315-4819-9537-8622a6ca1fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:56.986131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1069/2138] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:56.993751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a90285-ef46-48c0-8df7-6707e340f1af None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:56.994421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a90285-ef46-48c0-8df7-6707e340f1af None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:56.994842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a90285-ef46-48c0-8df7-6707e340f1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.035172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-78a90285-ef46-48c0-8df7-6707e340f1af None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:57.036585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-78a90285-ef46-48c0-8df7-6707e340f1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:57.052137 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c93b8-3315-4819-9537-8622a6ca1fef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.054618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1070/2139] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:57.067128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c93b8-3315-4819-9537-8622a6ca1fef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.068677 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fac1b6-0503-48eb-b383-235d70517fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.068865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fac1b6-0503-48eb-b383-235d70517fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.069146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fac1b6-0503-48eb-b383-235d70517fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.069422 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fac1b6-0503-48eb-b383-235d70517fe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.070125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fac1b6-0503-48eb-b383-235d70517fe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 04:59:57.070426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fac1b6-0503-48eb-b383-235d70517fe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.072082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fac1b6-0503-48eb-b383-235d70517fe4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.086877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1070/2140] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 04:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 04:59:57.091162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a90285-ef46-48c0-8df7-6707e340f1af None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.104057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a90285-ef46-48c0-8df7-6707e340f1af None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.109475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993cd5d-5975-4eea-a8bc-4c2dccdad823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.110227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993cd5d-5975-4eea-a8bc-4c2dccdad823 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.110621 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993cd5d-5975-4eea-a8bc-4c2dccdad823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.110966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993cd5d-5975-4eea-a8bc-4c2dccdad823 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.111721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993cd5d-5975-4eea-a8bc-4c2dccdad823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.114266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993cd5d-5975-4eea-a8bc-4c2dccdad823 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.161339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1071/2141] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:57 2022] POST /identity/v3/projects => generated 357 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:57.174878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fac1b6-0503-48eb-b383-235d70517fe4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.185804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fac1b6-0503-48eb-b383-235d70517fe4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.260538 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a2050-3ab7-4b94-96f1-2dc5f614d5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.261438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a2050-3ab7-4b94-96f1-2dc5f614d5fc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.261573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a2050-3ab7-4b94-96f1-2dc5f614d5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.261841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a2050-3ab7-4b94-96f1-2dc5f614d5fc admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.262684 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a2050-3ab7-4b94-96f1-2dc5f614d5fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.264741 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a2050-3ab7-4b94-96f1-2dc5f614d5fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.291602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1071/2142] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:57 2022] POST /identity/v3/users => generated 279 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 04:59:57.295606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993cd5d-5975-4eea-a8bc-4c2dccdad823 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.309024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993cd5d-5975-4eea-a8bc-4c2dccdad823 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.313787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe5bd4-b208-4efd-9135-e4224019c3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.314400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe5bd4-b208-4efd-9135-e4224019c3de admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.314710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe5bd4-b208-4efd-9135-e4224019c3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.314979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe5bd4-b208-4efd-9135-e4224019c3de admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.316256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe5bd4-b208-4efd-9135-e4224019c3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.318293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe5bd4-b208-4efd-9135-e4224019c3de admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.335404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1072/2143] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:57.336206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a2050-3ab7-4b94-96f1-2dc5f614d5fc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.351938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a2050-3ab7-4b94-96f1-2dc5f614d5fc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.357502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a2cce-ebd6-431d-98ab-a7a538da91be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.358955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a2cce-ebd6-431d-98ab-a7a538da91be admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.359450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a2cce-ebd6-431d-98ab-a7a538da91be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.360074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a2cce-ebd6-431d-98ab-a7a538da91be admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.361150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a2cce-ebd6-431d-98ab-a7a538da91be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.363095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a2cce-ebd6-431d-98ab-a7a538da91be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.379398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1072/2144] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:57.380851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe5bd4-b208-4efd-9135-e4224019c3de admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.391898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe5bd4-b208-4efd-9135-e4224019c3de admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.397509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba039b-2e2c-44f7-b471-ab351b40695b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.398257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba039b-2e2c-44f7-b471-ab351b40695b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.398744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba039b-2e2c-44f7-b471-ab351b40695b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.398923 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba039b-2e2c-44f7-b471-ab351b40695b admin admin] PATH_INFO: `/v3/projects/497d9d6d607c48fab65b3604b24806af/users/6016a68384714afc9a6eba092a834850/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.416040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba039b-2e2c-44f7-b471-ab351b40695b admin admin] RBAC: Authorizing `identity:create_grant(project_id=497d9d6d607c48fab65b3604b24806af, user_id=6016a68384714afc9a6eba092a834850, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.419128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba039b-2e2c-44f7-b471-ab351b40695b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.434021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1073/2145] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:57 2022] PUT /identity/v3/projects/497d9d6d607c48fab65b3604b24806af/users/6016a68384714afc9a6eba092a834850/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:57.447642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6f985-31cf-4c28-99e5-865ba002f056 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.447937 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6f985-31cf-4c28-99e5-865ba002f056 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.448209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6f985-31cf-4c28-99e5-865ba002f056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.479370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a3c6f985-31cf-4c28-99e5-865ba002f056 None None] MFA Rules not processed for user `6016a68384714afc9a6eba092a834850`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:57.480897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3c6f985-31cf-4c28-99e5-865ba002f056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:57.538011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:57.563333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1073/2146] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:57 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:57.568650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba039b-2e2c-44f7-b471-ab351b40695b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.657754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba039b-2e2c-44f7-b471-ab351b40695b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.664300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a1a8a-96a5-479d-8e3b-4ce04fa1f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.665650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a1a8a-96a5-479d-8e3b-4ce04fa1f249 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.666112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a1a8a-96a5-479d-8e3b-4ce04fa1f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.666394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a1a8a-96a5-479d-8e3b-4ce04fa1f249 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.667158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a1a8a-96a5-479d-8e3b-4ce04fa1f249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.669524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a1a8a-96a5-479d-8e3b-4ce04fa1f249 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.712182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1074/2147] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 04:59:57 2022] POST /identity/v3/projects => generated 357 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:57.716900 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6f985-31cf-4c28-99e5-865ba002f056 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.728977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6f985-31cf-4c28-99e5-865ba002f056 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.828868 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfb812-97e8-44ee-b594-b763e987f866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.829528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfb812-97e8-44ee-b594-b763e987f866 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.829818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfb812-97e8-44ee-b594-b763e987f866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.830091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfb812-97e8-44ee-b594-b763e987f866 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.830897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfb812-97e8-44ee-b594-b763e987f866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.833261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfb812-97e8-44ee-b594-b763e987f866 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.868183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1074/2148] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 04:59:57 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 04:59:57.874628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a1a8a-96a5-479d-8e3b-4ce04fa1f249 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.884421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a1a8a-96a5-479d-8e3b-4ce04fa1f249 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.890118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175d232-02f0-480e-b928-f40346311c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.891035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175d232-02f0-480e-b928-f40346311c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.891470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175d232-02f0-480e-b928-f40346311c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.891909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175d232-02f0-480e-b928-f40346311c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.893142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175d232-02f0-480e-b928-f40346311c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.895071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175d232-02f0-480e-b928-f40346311c3c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.916768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1075/2149] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 04:59:57.922005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfb812-97e8-44ee-b594-b763e987f866 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:57.936374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfb812-97e8-44ee-b594-b763e987f866 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:57.943240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d617e-93a8-4311-bde8-fa7c60cf37db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:57.944248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d617e-93a8-4311-bde8-fa7c60cf37db admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:57.944611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d617e-93a8-4311-bde8-fa7c60cf37db admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:57.944894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d617e-93a8-4311-bde8-fa7c60cf37db admin admin] PATH_INFO: `/v3/projects/9f3dad2790714286bbc7b157afb2e65c/users/fd01d1225b134c75b3f69f18a3999037/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:57.974258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d617e-93a8-4311-bde8-fa7c60cf37db admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3dad2790714286bbc7b157afb2e65c, user_id=fd01d1225b134c75b3f69f18a3999037, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:57.977308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d617e-93a8-4311-bde8-fa7c60cf37db admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:57.998473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1075/2150] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:57 2022] PUT /identity/v3/projects/9f3dad2790714286bbc7b157afb2e65c/users/fd01d1225b134c75b3f69f18a3999037/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:57.999461 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175d232-02f0-480e-b928-f40346311c3c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:58.011947 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175d232-02f0-480e-b928-f40346311c3c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:58.064250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:58.098792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d0302-54f3-4258-941e-3d8973ff55b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:58.099511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d0302-54f3-4258-941e-3d8973ff55b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:58.099851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d0302-54f3-4258-941e-3d8973ff55b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:58.100115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d0302-54f3-4258-941e-3d8973ff55b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:58.100871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d0302-54f3-4258-941e-3d8973ff55b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:58.102677 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d0302-54f3-4258-941e-3d8973ff55b7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:58.123157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1076/2151] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 04:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 04:59:58.127905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d617e-93a8-4311-bde8-fa7c60cf37db admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:58.154292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d617e-93a8-4311-bde8-fa7c60cf37db admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:58.162521 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedba25f-0883-44fd-8e75-04a02759b3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:58.163248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedba25f-0883-44fd-8e75-04a02759b3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:58.163530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedba25f-0883-44fd-8e75-04a02759b3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:58.163806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedba25f-0883-44fd-8e75-04a02759b3b8 admin admin] PATH_INFO: `/v3/projects/9f3dad2790714286bbc7b157afb2e65c/users/fd01d1225b134c75b3f69f18a3999037/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:58.171560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedba25f-0883-44fd-8e75-04a02759b3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3dad2790714286bbc7b157afb2e65c, user_id=fd01d1225b134c75b3f69f18a3999037, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:58.176717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedba25f-0883-44fd-8e75-04a02759b3b8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:58.193047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1076/2152] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 04:59:58 2022] PUT /identity/v3/projects/9f3dad2790714286bbc7b157afb2e65c/users/fd01d1225b134c75b3f69f18a3999037/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:58.207815 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778348b0-0049-4067-899d-f8f82c8c4741 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:58.207815 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778348b0-0049-4067-899d-f8f82c8c4741 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:58.207815 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778348b0-0049-4067-899d-f8f82c8c4741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:58.255889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-778348b0-0049-4067-899d-f8f82c8c4741 None None] MFA Rules not processed for user `fd01d1225b134c75b3f69f18a3999037`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:58.257442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-778348b0-0049-4067-899d-f8f82c8c4741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:58.391343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1077/2153] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:58 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:58.405952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedba25f-0883-44fd-8e75-04a02759b3b8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:58.408035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bedba25f-0883-44fd-8e75-04a02759b3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:58.502836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedba25f-0883-44fd-8e75-04a02759b3b8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:58.515536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec511a3-029a-4e3e-b444-bc4b03d62528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:58.516282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec511a3-029a-4e3e-b444-bc4b03d62528 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:58.516630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec511a3-029a-4e3e-b444-bc4b03d62528 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:58.516986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec511a3-029a-4e3e-b444-bc4b03d62528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:58.529804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec511a3-029a-4e3e-b444-bc4b03d62528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:58.531317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec511a3-029a-4e3e-b444-bc4b03d62528 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:58.544934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1077/2154] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:58 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:58.639224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778348b0-0049-4067-899d-f8f82c8c4741 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:58.661983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778348b0-0049-4067-899d-f8f82c8c4741 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:58.670704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640bb8f-79cb-4c3d-96b8-fbcb4e88feba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:58.671417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640bb8f-79cb-4c3d-96b8-fbcb4e88feba service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:58.671806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640bb8f-79cb-4c3d-96b8-fbcb4e88feba service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:58.672144 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640bb8f-79cb-4c3d-96b8-fbcb4e88feba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:58.802762 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640bb8f-79cb-4c3d-96b8-fbcb4e88feba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:58.804419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640bb8f-79cb-4c3d-96b8-fbcb4e88feba service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:58.822591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1078/2155] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:58 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 04:59:59.526736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec511a3-029a-4e3e-b444-bc4b03d62528 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:59.528303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dec511a3-029a-4e3e-b444-bc4b03d62528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:59.576217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:59.604786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec511a3-029a-4e3e-b444-bc4b03d62528 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:59.610356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:59.611091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:59.611378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:59.611661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] PATH_INFO: `/v3/users/093c939da58448468abdda19067ad523` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:59.613437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=093c939da58448468abdda19067ad523)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:59.615411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:59.671021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '093c939da58448468abdda19067ad523'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.679158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '093c939da58448468abdda19067ad523'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.679864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '093c939da58448468abdda19067ad523'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.680485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '093c939da58448468abdda19067ad523'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.693142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '093c939da58448468abdda19067ad523'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.708504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1078/2156] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:59 2022] DELETE /identity/v3/users/093c939da58448468abdda19067ad523 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 04:59:59.713344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b155f39-c8fc-406a-ac2b-32754d3065cf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:59.713344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b155f39-c8fc-406a-ac2b-32754d3065cf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:59.713602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b155f39-c8fc-406a-ac2b-32754d3065cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:59.723026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:59.725806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:59.741708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7b155f39-c8fc-406a-ac2b-32754d3065cf None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 04:59:59.743052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7b155f39-c8fc-406a-ac2b-32754d3065cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:59.805598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:59.812948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88c52b-e162-43c3-a538-45c0f2c0e851 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:59.817092 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1079/2157] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 04:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 04:59:59.818074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:59.818817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:59.819114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:59.819378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] PATH_INFO: `/v3/users/711a2ec788b341eca224a84c288406b1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:59.820571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=711a2ec788b341eca224a84c288406b1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:59.822558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:59.827797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b155f39-c8fc-406a-ac2b-32754d3065cf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 04:59:59.829297 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7b155f39-c8fc-406a-ac2b-32754d3065cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:59.878938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '711a2ec788b341eca224a84c288406b1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.886559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '711a2ec788b341eca224a84c288406b1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.887348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '711a2ec788b341eca224a84c288406b1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.888044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '711a2ec788b341eca224a84c288406b1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.894148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 04:59:59.899868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b155f39-c8fc-406a-ac2b-32754d3065cf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 04:59:59.904560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf2369-ec1c-4902-9ee5-82185540d5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 04:59:59.905362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf2369-ec1c-4902-9ee5-82185540d5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 04:59:59.905477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf2369-ec1c-4902-9ee5-82185540d5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 04:59:59.905616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '711a2ec788b341eca224a84c288406b1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 04:59:59.905730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf2369-ec1c-4902-9ee5-82185540d5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 04:59:59.907930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f4bf2369-ec1c-4902-9ee5-82185540d5f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 04:59:59.925154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1079/2158] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 04:59:59 2022] DELETE /identity/v3/users/711a2ec788b341eca224a84c288406b1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 04:59:59.925438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf2369-ec1c-4902-9ee5-82185540d5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 04:59:59.925438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf2369-ec1c-4902-9ee5-82185540d5f6 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 04:59:59.936078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1080/2159] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 04:59:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:00:00.172711 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:00.174266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:00.259261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c295f6-9a72-424f-a0c5-72c7eed99de9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:00.264975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:00.265904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:00.266250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:00.266637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] PATH_INFO: `/v3/projects/6d00250cfb0f45e6bbb74e56422d2db2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:00.268252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d00250cfb0f45e6bbb74e56422d2db2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:00.270869 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:00.326558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d00250cfb0f45e6bbb74e56422d2db2', 'request_id': 'req-f50ce7c4-d7ba-4735-bc76-151a52e75598'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.335408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d00250cfb0f45e6bbb74e56422d2db2', 'request_id': 'req-f50ce7c4-d7ba-4735-bc76-151a52e75598'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.341336 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d00250cfb0f45e6bbb74e56422d2db2', 'request_id': 'req-f50ce7c4-d7ba-4735-bc76-151a52e75598'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.356069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] The token cache is being invalidate because project 6d00250cfb0f45e6bbb74e56422d2db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:00.356366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.357186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.360093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1080/2160] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:00 2022] DELETE /identity/v3/projects/6d00250cfb0f45e6bbb74e56422d2db2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:00.367517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf2369-ec1c-4902-9ee5-82185540d5f6 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:00.383377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf2369-ec1c-4902-9ee5-82185540d5f6 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:00.433042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:00.434950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:00.488035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:00.488985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:00.490473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:00.490473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] PATH_INFO: `/v3/users/f1ccaae8c452440bbe6eb57ec5069c3c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:00.493396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ccaae8c452440bbe6eb57ec5069c3c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:00.495226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:00.509053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ce7c4-d7ba-4735-bc76-151a52e75598 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:00.514936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:00.516508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:00.516942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:00.517362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] PATH_INFO: `/v3/projects/b814b33e4f2f4a84ab57e7588bc0714e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:00.520022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b814b33e4f2f4a84ab57e7588bc0714e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:00.522241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:00.536215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:00.556366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ccaae8c452440bbe6eb57ec5069c3c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.565092 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ccaae8c452440bbe6eb57ec5069c3c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.565868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ccaae8c452440bbe6eb57ec5069c3c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.566515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ccaae8c452440bbe6eb57ec5069c3c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.582141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b814b33e4f2f4a84ab57e7588bc0714e', 'request_id': 'req-e8a9dd55-d873-44b9-b33c-adb97d32347a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.583099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6054ca4b-2359-4cff-95fa-c9d378237a0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ccaae8c452440bbe6eb57ec5069c3c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.590424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b814b33e4f2f4a84ab57e7588bc0714e', 'request_id': 'req-e8a9dd55-d873-44b9-b33c-adb97d32347a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.598037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b814b33e4f2f4a84ab57e7588bc0714e', 'request_id': 'req-e8a9dd55-d873-44b9-b33c-adb97d32347a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.599826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1081/2161] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:00 2022] DELETE /identity/v3/users/f1ccaae8c452440bbe6eb57ec5069c3c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:00.610701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] The token cache is being invalidate because project b814b33e4f2f4a84ab57e7588bc0714e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:00.610998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.612129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e8a9dd55-d873-44b9-b33c-adb97d32347a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.613578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1081/2162] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:00 2022] DELETE /identity/v3/projects/b814b33e4f2f4a84ab57e7588bc0714e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:00.619204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438452-e65f-4058-9c4b-fa0a6911fcde None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:00.619699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438452-e65f-4058-9c4b-fa0a6911fcde None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:00.620050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438452-e65f-4058-9c4b-fa0a6911fcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:00.640748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8e438452-e65f-4058-9c4b-fa0a6911fcde None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:00.642135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8e438452-e65f-4058-9c4b-fa0a6911fcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:00.672796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:00.727173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1082/2163] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:00.734989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410637-e440-439c-8989-b8ff74f44ee5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:00.735285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410637-e440-439c-8989-b8ff74f44ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:00.735567 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410637-e440-439c-8989-b8ff74f44ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:00.757376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-00410637-e440-439c-8989-b8ff74f44ee5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:00.757975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-00410637-e440-439c-8989-b8ff74f44ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:00.770932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1082/2164] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:00.772327 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438452-e65f-4058-9c4b-fa0a6911fcde None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:00.783405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438452-e65f-4058-9c4b-fa0a6911fcde None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:00.792277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:00.794065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:00.794616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:00.794876 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:00.795916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:00.796274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:00.799048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:00.820079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1083/2165] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:00.824822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410637-e440-439c-8989-b8ff74f44ee5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:00.835809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:00.839057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:00.850899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410637-e440-439c-8989-b8ff74f44ee5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:00.856297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851f31e-17f5-4e25-acda-f1ffc6501ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:00.857227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851f31e-17f5-4e25-acda-f1ffc6501ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:00.857429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851f31e-17f5-4e25-acda-f1ffc6501ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:00.857807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851f31e-17f5-4e25-acda-f1ffc6501ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:00.858624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851f31e-17f5-4e25-acda-f1ffc6501ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:00.860721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851f31e-17f5-4e25-acda-f1ffc6501ed3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:00.872695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18442-c4a0-41a6-bdb1-a164898e7a21 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:00.879480 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:00.880191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:00.880532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:00.880812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] PATH_INFO: `/v3/projects/7ad312edbd634268b08df7b02c842d15` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:00.882039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad312edbd634268b08df7b02c842d15)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:00.884099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:00.899337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1083/2166] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:00 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:00.903438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851f31e-17f5-4e25-acda-f1ffc6501ed3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:00.958506 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad312edbd634268b08df7b02c842d15', 'request_id': 'req-6707a908-e095-4195-b8ab-3100e9bbe372'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.967554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad312edbd634268b08df7b02c842d15', 'request_id': 'req-6707a908-e095-4195-b8ab-3100e9bbe372'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.979572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad312edbd634268b08df7b02c842d15', 'request_id': 'req-6707a908-e095-4195-b8ab-3100e9bbe372'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.984093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851f31e-17f5-4e25-acda-f1ffc6501ed3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:00.994818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] The token cache is being invalidate because project 7ad312edbd634268b08df7b02c842d15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:00.995178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.995956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6707a908-e095-4195-b8ab-3100e9bbe372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:00.997626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1084/2167] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:00 2022] DELETE /identity/v3/projects/7ad312edbd634268b08df7b02c842d15 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:01.002251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4921b4f-7834-4288-9233-953542022989 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.003823 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4921b4f-7834-4288-9233-953542022989 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.004093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4921b4f-7834-4288-9233-953542022989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.041789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a4921b4f-7834-4288-9233-953542022989 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:01.043284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a4921b4f-7834-4288-9233-953542022989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:01.044809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6c471-59d7-4760-b96b-f0052fad3b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.045505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6c471-59d7-4760-b96b-f0052fad3b70 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.045865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6c471-59d7-4760-b96b-f0052fad3b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.046211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6c471-59d7-4760-b96b-f0052fad3b70 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.046980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6c471-59d7-4760-b96b-f0052fad3b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.048988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6c471-59d7-4760-b96b-f0052fad3b70 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.053614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1085/2168] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:01.060484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5605e-8d3d-4621-bbf2-7306b7bf1099 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.060995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5605e-8d3d-4621-bbf2-7306b7bf1099 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.062003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5605e-8d3d-4621-bbf2-7306b7bf1099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.085927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1084/2169] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:00 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:01.086463 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2da5605e-8d3d-4621-bbf2-7306b7bf1099 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:01.087069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6c471-59d7-4760-b96b-f0052fad3b70 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.088465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2da5605e-8d3d-4621-bbf2-7306b7bf1099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:01.088754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5e6c471-59d7-4760-b96b-f0052fad3b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:01.098267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1086/2170] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:01.101989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5605e-8d3d-4621-bbf2-7306b7bf1099 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.104490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6c471-59d7-4760-b96b-f0052fad3b70 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.109255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73191d73-1a02-4779-b936-eddc579eb159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.109874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73191d73-1a02-4779-b936-eddc579eb159 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.110163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73191d73-1a02-4779-b936-eddc579eb159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.110474 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73191d73-1a02-4779-b936-eddc579eb159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.111183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73191d73-1a02-4779-b936-eddc579eb159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.113585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73191d73-1a02-4779-b936-eddc579eb159 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.115234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:01.118147 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5605e-8d3d-4621-bbf2-7306b7bf1099 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.124809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d377404-613c-42c2-9064-e386c4dfbac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.125434 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d377404-613c-42c2-9064-e386c4dfbac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.126008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1085/2171] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:01.126685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d377404-613c-42c2-9064-e386c4dfbac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.126986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d377404-613c-42c2-9064-e386c4dfbac3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.128140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d377404-613c-42c2-9064-e386c4dfbac3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:01.128140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d377404-613c-42c2-9064-e386c4dfbac3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.129836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73191d73-1a02-4779-b936-eddc579eb159 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.130142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d377404-613c-42c2-9064-e386c4dfbac3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.140677 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73191d73-1a02-4779-b936-eddc579eb159 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.146396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1087/2172] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 05:00:01.152143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d377404-613c-42c2-9064-e386c4dfbac3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.154226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff94f9-956d-4f3d-8135-d340d2ba409f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.154891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff94f9-956d-4f3d-8135-d340d2ba409f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.155178 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff94f9-956d-4f3d-8135-d340d2ba409f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.155454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff94f9-956d-4f3d-8135-d340d2ba409f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.156154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff94f9-956d-4f3d-8135-d340d2ba409f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.157792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff94f9-956d-4f3d-8135-d340d2ba409f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.169428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d377404-613c-42c2-9064-e386c4dfbac3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.175579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266801d3-0dd2-4c86-8e9e-7e38678faf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.176237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266801d3-0dd2-4c86-8e9e-7e38678faf6a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.176594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266801d3-0dd2-4c86-8e9e-7e38678faf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.176868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266801d3-0dd2-4c86-8e9e-7e38678faf6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.177558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266801d3-0dd2-4c86-8e9e-7e38678faf6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.179486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266801d3-0dd2-4c86-8e9e-7e38678faf6a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.181919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1086/2173] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:01.185800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff94f9-956d-4f3d-8135-d340d2ba409f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.197119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff94f9-956d-4f3d-8135-d340d2ba409f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.202307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf4f47-4958-4e39-8e07-8a4aa60041cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.204040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf4f47-4958-4e39-8e07-8a4aa60041cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.204339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf4f47-4958-4e39-8e07-8a4aa60041cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.204623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf4f47-4958-4e39-8e07-8a4aa60041cd admin admin] PATH_INFO: `/v3/projects/14ab99750fff4fd287661c1586894b08/users/ed1db580dbe94a02ad4ddb9be9c29032/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.225300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1088/2174] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:01.226651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266801d3-0dd2-4c86-8e9e-7e38678faf6a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.253061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf4f47-4958-4e39-8e07-8a4aa60041cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ab99750fff4fd287661c1586894b08, user_id=ed1db580dbe94a02ad4ddb9be9c29032, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.257328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf4f47-4958-4e39-8e07-8a4aa60041cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.274918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1087/2175] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:01 2022] PUT /identity/v3/projects/14ab99750fff4fd287661c1586894b08/users/ed1db580dbe94a02ad4ddb9be9c29032/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:01.286650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:01.286863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ae5c-8ee5-4434-9801-5d94f1190153 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.287031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ae5c-8ee5-4434-9801-5d94f1190153 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.287307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246ae5c-8ee5-4434-9801-5d94f1190153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.308292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266801d3-0dd2-4c86-8e9e-7e38678faf6a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.317488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4246ae5c-8ee5-4434-9801-5d94f1190153 None None] MFA Rules not processed for user `ed1db580dbe94a02ad4ddb9be9c29032`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:01.319285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4246ae5c-8ee5-4434-9801-5d94f1190153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:01.371544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca854c1-ddd5-41bd-8380-5f3050c31e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.372546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca854c1-ddd5-41bd-8380-5f3050c31e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.372835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca854c1-ddd5-41bd-8380-5f3050c31e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.373226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca854c1-ddd5-41bd-8380-5f3050c31e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.373902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca854c1-ddd5-41bd-8380-5f3050c31e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.375857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca854c1-ddd5-41bd-8380-5f3050c31e2d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.400371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1089/2176] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:01.404840 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca854c1-ddd5-41bd-8380-5f3050c31e2d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.407779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1088/2177] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:01.414001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ae5c-8ee5-4434-9801-5d94f1190153 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.421722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca854c1-ddd5-41bd-8380-5f3050c31e2d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.426899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70852cfc-14bd-4846-820d-2539acd9b467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.427790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70852cfc-14bd-4846-820d-2539acd9b467 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.428099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70852cfc-14bd-4846-820d-2539acd9b467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.428331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70852cfc-14bd-4846-820d-2539acd9b467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.429002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246ae5c-8ee5-4434-9801-5d94f1190153 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.429095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70852cfc-14bd-4846-820d-2539acd9b467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.431014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70852cfc-14bd-4846-820d-2539acd9b467 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.435426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49469f-f82d-4ea5-a47d-24fa5fb0615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.436562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49469f-f82d-4ea5-a47d-24fa5fb0615f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.436562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49469f-f82d-4ea5-a47d-24fa5fb0615f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.437648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49469f-f82d-4ea5-a47d-24fa5fb0615f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.438583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49469f-f82d-4ea5-a47d-24fa5fb0615f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.445820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1090/2178] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:01.447242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70852cfc-14bd-4846-820d-2539acd9b467 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.451693 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49469f-f82d-4ea5-a47d-24fa5fb0615f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.463008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70852cfc-14bd-4846-820d-2539acd9b467 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.472390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f07a14-4a94-4463-b543-962316bcf8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.473008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f07a14-4a94-4463-b543-962316bcf8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.473330 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f07a14-4a94-4463-b543-962316bcf8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.473603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f07a14-4a94-4463-b543-962316bcf8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.474332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f07a14-4a94-4463-b543-962316bcf8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.476040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f07a14-4a94-4463-b543-962316bcf8b9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.487900 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1091/2179] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:01.489220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1089/2180] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:01.492010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f07a14-4a94-4463-b543-962316bcf8b9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.494486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49469f-f82d-4ea5-a47d-24fa5fb0615f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.503445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f07a14-4a94-4463-b543-962316bcf8b9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.567846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c24cdb-5d9a-431c-b73b-829fe7c1e0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.568667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c24cdb-5d9a-431c-b73b-829fe7c1e0ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.569043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c24cdb-5d9a-431c-b73b-829fe7c1e0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.569417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c24cdb-5d9a-431c-b73b-829fe7c1e0ad admin admin] PATH_INFO: `/v3/projects/b1092087b96b4ed39814eeb89da4b0f1/users/28c687f86caa4cc99565c79e8b2c0aef/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.571513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49469f-f82d-4ea5-a47d-24fa5fb0615f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.577902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348284f-4fbc-49c7-82bf-07fa118d75c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.578559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348284f-4fbc-49c7-82bf-07fa118d75c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.578862 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348284f-4fbc-49c7-82bf-07fa118d75c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.579120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348284f-4fbc-49c7-82bf-07fa118d75c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.582312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348284f-4fbc-49c7-82bf-07fa118d75c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.582312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348284f-4fbc-49c7-82bf-07fa118d75c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.589474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c24cdb-5d9a-431c-b73b-829fe7c1e0ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1092087b96b4ed39814eeb89da4b0f1, user_id=28c687f86caa4cc99565c79e8b2c0aef, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.593104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c24cdb-5d9a-431c-b73b-829fe7c1e0ad admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.610254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1090/2181] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:01.611478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348284f-4fbc-49c7-82bf-07fa118d75c3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.623113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1092/2182] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:01 2022] PUT /identity/v3/projects/b1092087b96b4ed39814eeb89da4b0f1/users/28c687f86caa4cc99565c79e8b2c0aef/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:01.635742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a4511-b7a4-436c-87e1-1d7001662e2b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.636031 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a4511-b7a4-436c-87e1-1d7001662e2b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.636298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a4511-b7a4-436c-87e1-1d7001662e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.640253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348284f-4fbc-49c7-82bf-07fa118d75c3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.663349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:01.673713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1f1a4511-b7a4-436c-87e1-1d7001662e2b None None] MFA Rules not processed for user `28c687f86caa4cc99565c79e8b2c0aef`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:01.676254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1f1a4511-b7a4-436c-87e1-1d7001662e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:01.712653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:01.726455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46469df8-f018-4544-a9b6-4ce690be55e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.727101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46469df8-f018-4544-a9b6-4ce690be55e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.727446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46469df8-f018-4544-a9b6-4ce690be55e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.727785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46469df8-f018-4544-a9b6-4ce690be55e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.728549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46469df8-f018-4544-a9b6-4ce690be55e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.730242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46469df8-f018-4544-a9b6-4ce690be55e9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.740819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1091/2183] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:01.744709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46469df8-f018-4544-a9b6-4ce690be55e9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.751273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1093/2184] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:01.754277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46469df8-f018-4544-a9b6-4ce690be55e9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.755878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a4511-b7a4-436c-87e1-1d7001662e2b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.758954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b906045-6d9c-4c18-a09c-ed5e6208c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.759549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b906045-6d9c-4c18-a09c-ed5e6208c73b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.759827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b906045-6d9c-4c18-a09c-ed5e6208c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.760099 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b906045-6d9c-4c18-a09c-ed5e6208c73b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.760783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b906045-6d9c-4c18-a09c-ed5e6208c73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.762361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b906045-6d9c-4c18-a09c-ed5e6208c73b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.766731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a4511-b7a4-436c-87e1-1d7001662e2b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.771797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2b87a-d8dd-44cc-b76e-4110f133e998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.772408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2b87a-d8dd-44cc-b76e-4110f133e998 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.772690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2b87a-d8dd-44cc-b76e-4110f133e998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.773085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2b87a-d8dd-44cc-b76e-4110f133e998 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.773905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2b87a-d8dd-44cc-b76e-4110f133e998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.774030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1092/2185] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:01.775704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2b87a-d8dd-44cc-b76e-4110f133e998 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.778149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b906045-6d9c-4c18-a09c-ed5e6208c73b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.790113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b906045-6d9c-4c18-a09c-ed5e6208c73b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.795785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb09920-817b-453f-a039-8471823881c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.796536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb09920-817b-453f-a039-8471823881c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.796873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb09920-817b-453f-a039-8471823881c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.797150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb09920-817b-453f-a039-8471823881c6 admin admin] PATH_INFO: `/v3/projects/fdf2ff1b5d5b42318807917f3f3f8eda/users/87973478b5124cb9b999c29b3d10c510/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.807246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1094/2186] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:01.811804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2b87a-d8dd-44cc-b76e-4110f133e998 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.812785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb09920-817b-453f-a039-8471823881c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf2ff1b5d5b42318807917f3f3f8eda, user_id=87973478b5124cb9b999c29b3d10c510, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.816057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb09920-817b-453f-a039-8471823881c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.829493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1093/2187] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:01 2022] PUT /identity/v3/projects/fdf2ff1b5d5b42318807917f3f3f8eda/users/87973478b5124cb9b999c29b3d10c510/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:01.840088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a02e72-16ff-498c-81af-15bea6d29a6d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.840375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a02e72-16ff-498c-81af-15bea6d29a6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.840649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a02e72-16ff-498c-81af-15bea6d29a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.870783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d2a02e72-16ff-498c-81af-15bea6d29a6d None None] MFA Rules not processed for user `87973478b5124cb9b999c29b3d10c510`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:01.872041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d2a02e72-16ff-498c-81af-15bea6d29a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:01.882834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2b87a-d8dd-44cc-b76e-4110f133e998 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.938182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1094/2188] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:01.940810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade65b7-20ad-48ad-a34c-4701c11e15f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.941435 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade65b7-20ad-48ad-a34c-4701c11e15f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.941943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade65b7-20ad-48ad-a34c-4701c11e15f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.942082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade65b7-20ad-48ad-a34c-4701c11e15f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.942673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade65b7-20ad-48ad-a34c-4701c11e15f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.942849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a02e72-16ff-498c-81af-15bea6d29a6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.945196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade65b7-20ad-48ad-a34c-4701c11e15f1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.953431 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a02e72-16ff-498c-81af-15bea6d29a6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.958690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dcfb4-1fd9-48a0-8023-61e03ae24fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.959401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dcfb4-1fd9-48a0-8023-61e03ae24fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.959734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dcfb4-1fd9-48a0-8023-61e03ae24fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.960037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dcfb4-1fd9-48a0-8023-61e03ae24fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.960714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dcfb4-1fd9-48a0-8023-61e03ae24fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.963066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dcfb4-1fd9-48a0-8023-61e03ae24fb1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.969879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1095/2189] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:01.970880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade65b7-20ad-48ad-a34c-4701c11e15f1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:01.981295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade65b7-20ad-48ad-a34c-4701c11e15f1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:01.986258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2bfd11-7bae-4a7a-85d3-83ab8cb4438c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:01.986894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2bfd11-7bae-4a7a-85d3-83ab8cb4438c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:01.987731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2bfd11-7bae-4a7a-85d3-83ab8cb4438c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:01.988014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2bfd11-7bae-4a7a-85d3-83ab8cb4438c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:01.988681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2bfd11-7bae-4a7a-85d3-83ab8cb4438c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:01.990303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2bfd11-7bae-4a7a-85d3-83ab8cb4438c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:01.995017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1095/2190] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:01.995950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dcfb4-1fd9-48a0-8023-61e03ae24fb1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.001738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1096/2191] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:02.007363 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2bfd11-7bae-4a7a-85d3-83ab8cb4438c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.017028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2bfd11-7bae-4a7a-85d3-83ab8cb4438c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.051322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:02.060266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:02.082300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dcfb4-1fd9-48a0-8023-61e03ae24fb1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.087172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce7580-8518-4d4d-991a-217ec4185ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.087852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce7580-8518-4d4d-991a-217ec4185ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.088071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce7580-8518-4d4d-991a-217ec4185ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.088348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce7580-8518-4d4d-991a-217ec4185ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.088744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b141df-1d75-47dd-9b37-b7e23c6b6c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.089016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce7580-8518-4d4d-991a-217ec4185ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.089410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b141df-1d75-47dd-9b37-b7e23c6b6c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.089691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b141df-1d75-47dd-9b37-b7e23c6b6c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.089966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b141df-1d75-47dd-9b37-b7e23c6b6c80 admin admin] PATH_INFO: `/v3/projects/774b5e9de7a74c97b52bcaae2d734c03/users/3712fea816704621bb402b96cb0aceab/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.090968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce7580-8518-4d4d-991a-217ec4185ab0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.116214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b141df-1d75-47dd-9b37-b7e23c6b6c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774b5e9de7a74c97b52bcaae2d734c03, user_id=3712fea816704621bb402b96cb0aceab, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.117876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1096/2192] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:01 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:02.122249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce7580-8518-4d4d-991a-217ec4185ab0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.123297 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b141df-1d75-47dd-9b37-b7e23c6b6c80 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.133440 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce7580-8518-4d4d-991a-217ec4185ab0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.139687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1097/2193] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:02 2022] PUT /identity/v3/projects/774b5e9de7a74c97b52bcaae2d734c03/users/3712fea816704621bb402b96cb0aceab/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:02.143990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b141df-1d75-47dd-9b37-b7e23c6b6c80 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.202751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f375c-61c9-46a3-9caa-f89017f5a891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.203383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f375c-61c9-46a3-9caa-f89017f5a891 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.203663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f375c-61c9-46a3-9caa-f89017f5a891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.203933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f375c-61c9-46a3-9caa-f89017f5a891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.204653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f375c-61c9-46a3-9caa-f89017f5a891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.207013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f375c-61c9-46a3-9caa-f89017f5a891 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.212428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b141df-1d75-47dd-9b37-b7e23c6b6c80 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.222570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622076b-4ea7-4a97-b726-01b5ed7c7eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.222570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622076b-4ea7-4a97-b726-01b5ed7c7eac admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.222570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622076b-4ea7-4a97-b726-01b5ed7c7eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.222570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622076b-4ea7-4a97-b726-01b5ed7c7eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.223018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1097/2194] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:02.223118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622076b-4ea7-4a97-b726-01b5ed7c7eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.223118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7622076b-4ea7-4a97-b726-01b5ed7c7eac admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.224252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f375c-61c9-46a3-9caa-f89017f5a891 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.233943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1098/2195] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:02.238138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622076b-4ea7-4a97-b726-01b5ed7c7eac admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.238553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f375c-61c9-46a3-9caa-f89017f5a891 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.243059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6e24c-0d69-452c-9562-b26fd66f872b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.243713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6e24c-0d69-452c-9562-b26fd66f872b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.243999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6e24c-0d69-452c-9562-b26fd66f872b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.244278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6e24c-0d69-452c-9562-b26fd66f872b admin admin] PATH_INFO: `/v3/projects/e54998b0463643e79c4d787c469987c4/users/fce97384b9f545ae864fddf2d2a59cb6/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.247916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622076b-4ea7-4a97-b726-01b5ed7c7eac admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.252661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf8113-15b6-4923-a80e-bcc92b8efd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.253428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf8113-15b6-4923-a80e-bcc92b8efd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.253712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf8113-15b6-4923-a80e-bcc92b8efd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.253986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf8113-15b6-4923-a80e-bcc92b8efd0e admin admin] PATH_INFO: `/v3/projects/774b5e9de7a74c97b52bcaae2d734c03/users/3712fea816704621bb402b96cb0aceab/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.256031 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf8113-15b6-4923-a80e-bcc92b8efd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=774b5e9de7a74c97b52bcaae2d734c03, user_id=3712fea816704621bb402b96cb0aceab, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.258750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf8113-15b6-4923-a80e-bcc92b8efd0e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.261639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6e24c-0d69-452c-9562-b26fd66f872b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e54998b0463643e79c4d787c469987c4, user_id=fce97384b9f545ae864fddf2d2a59cb6, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.265055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6e24c-0d69-452c-9562-b26fd66f872b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.272630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1099/2196] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:02 2022] PUT /identity/v3/projects/774b5e9de7a74c97b52bcaae2d734c03/users/3712fea816704621bb402b96cb0aceab/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:02.280783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1098/2197] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:02 2022] PUT /identity/v3/projects/e54998b0463643e79c4d787c469987c4/users/fce97384b9f545ae864fddf2d2a59cb6/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:02.283581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62135998-67a2-4ecb-a705-55665c23e69b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.283719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62135998-67a2-4ecb-a705-55665c23e69b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.284872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62135998-67a2-4ecb-a705-55665c23e69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.285056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6e24c-0d69-452c-9562-b26fd66f872b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.314063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-62135998-67a2-4ecb-a705-55665c23e69b None None] MFA Rules not processed for user `3712fea816704621bb402b96cb0aceab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:02.315650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-62135998-67a2-4ecb-a705-55665c23e69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:02.357698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6e24c-0d69-452c-9562-b26fd66f872b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.363981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ac92-ccc1-457e-9c57-712a5f47678b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.364700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ac92-ccc1-457e-9c57-712a5f47678b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.365060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ac92-ccc1-457e-9c57-712a5f47678b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.365446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9ac92-ccc1-457e-9c57-712a5f47678b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.366465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9ac92-ccc1-457e-9c57-712a5f47678b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.371651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9ac92-ccc1-457e-9c57-712a5f47678b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.387210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1099/2198] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:02.391469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ac92-ccc1-457e-9c57-712a5f47678b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.405830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9ac92-ccc1-457e-9c57-712a5f47678b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.411769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085908d9-62b2-4dd3-b55c-3f46fb131da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.412752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085908d9-62b2-4dd3-b55c-3f46fb131da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.413765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1100/2199] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:02.416782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085908d9-62b2-4dd3-b55c-3f46fb131da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.417214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085908d9-62b2-4dd3-b55c-3f46fb131da6 admin admin] PATH_INFO: `/v3/projects/e54998b0463643e79c4d787c469987c4/users/fce97384b9f545ae864fddf2d2a59cb6/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.420294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085908d9-62b2-4dd3-b55c-3f46fb131da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e54998b0463643e79c4d787c469987c4, user_id=fce97384b9f545ae864fddf2d2a59cb6, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.423648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085908d9-62b2-4dd3-b55c-3f46fb131da6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.424556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62135998-67a2-4ecb-a705-55665c23e69b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.426185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-62135998-67a2-4ecb-a705-55665c23e69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:02.442241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1100/2200] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:02 2022] PUT /identity/v3/projects/e54998b0463643e79c4d787c469987c4/users/fce97384b9f545ae864fddf2d2a59cb6/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:02.454529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99f287-0888-422f-8142-a8113e62f2b8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.454830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99f287-0888-422f-8142-a8113e62f2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.455102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99f287-0888-422f-8142-a8113e62f2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.471766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:02.492388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ff99f287-0888-422f-8142-a8113e62f2b8 None None] MFA Rules not processed for user `fce97384b9f545ae864fddf2d2a59cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:02.493828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff99f287-0888-422f-8142-a8113e62f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:02.517001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62135998-67a2-4ecb-a705-55665c23e69b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.519727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:02.585064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1101/2201] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:02.589217 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71740a2-5b24-4bec-8ab5-6034a561a497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.590643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71740a2-5b24-4bec-8ab5-6034a561a497 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.591562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71740a2-5b24-4bec-8ab5-6034a561a497 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.591562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71740a2-5b24-4bec-8ab5-6034a561a497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.598263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99f287-0888-422f-8142-a8113e62f2b8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:02.599875 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff99f287-0888-422f-8142-a8113e62f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:02.612918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99f287-0888-422f-8142-a8113e62f2b8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:02.618589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f469864-d1d3-477c-9ee7-5e88de50cb69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:02.619207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f469864-d1d3-477c-9ee7-5e88de50cb69 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:02.619755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f469864-d1d3-477c-9ee7-5e88de50cb69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:02.620314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f469864-d1d3-477c-9ee7-5e88de50cb69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:02.660113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71740a2-5b24-4bec-8ab5-6034a561a497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.661617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71740a2-5b24-4bec-8ab5-6034a561a497 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.672542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1101/2202] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:00:02.688592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f469864-d1d3-477c-9ee7-5e88de50cb69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:02.690740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f469864-d1d3-477c-9ee7-5e88de50cb69 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:02.701160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1102/2203] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:00:04.302494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71740a2-5b24-4bec-8ab5-6034a561a497 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:04.311626 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71740a2-5b24-4bec-8ab5-6034a561a497 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:04.315781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ffa70-0990-40ee-ada5-4197b9565b7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:04.316395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ffa70-0990-40ee-ada5-4197b9565b7f service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:04.316672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ffa70-0990-40ee-ada5-4197b9565b7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:04.316941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ffa70-0990-40ee-ada5-4197b9565b7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:04.375265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ffa70-0990-40ee-ada5-4197b9565b7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:04.377006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ffa70-0990-40ee-ada5-4197b9565b7f service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:04.386530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1102/2204] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:05.165934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f469864-d1d3-477c-9ee7-5e88de50cb69 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:05.175361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f469864-d1d3-477c-9ee7-5e88de50cb69 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:05.179331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2080a-7566-4440-b0fa-82fa4d77c024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:05.179975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2080a-7566-4440-b0fa-82fa4d77c024 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:05.180259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2080a-7566-4440-b0fa-82fa4d77c024 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:05.180535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2080a-7566-4440-b0fa-82fa4d77c024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:05.189698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2080a-7566-4440-b0fa-82fa4d77c024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:05.191795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2080a-7566-4440-b0fa-82fa4d77c024 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:05.201217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1103/2205] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:07.157557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ffa70-0990-40ee-ada5-4197b9565b7f service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:07.177062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ffa70-0990-40ee-ada5-4197b9565b7f service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:07.197440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a2aff-baf9-48ca-a162-a576117665c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:07.198184 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a2aff-baf9-48ca-a162-a576117665c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:07.198600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a2aff-baf9-48ca-a162-a576117665c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:07.198988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a2aff-baf9-48ca-a162-a576117665c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:07.247805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:07.319837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a2aff-baf9-48ca-a162-a576117665c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:07.321325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a2aff-baf9-48ca-a162-a576117665c9 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:07.333372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1103/2206] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:09.351092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2080a-7566-4440-b0fa-82fa4d77c024 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:09.428096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2080a-7566-4440-b0fa-82fa4d77c024 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:09.434981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:09.435837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:09.436086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:09.436360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] PATH_INFO: `/v3/users/ed1db580dbe94a02ad4ddb9be9c29032` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:09.437894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed1db580dbe94a02ad4ddb9be9c29032)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:09.440005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:09.513490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:09.516895 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed1db580dbe94a02ad4ddb9be9c29032'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:09.528232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed1db580dbe94a02ad4ddb9be9c29032'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:09.529127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed1db580dbe94a02ad4ddb9be9c29032'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:09.529915 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed1db580dbe94a02ad4ddb9be9c29032'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:09.544795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed1db580dbe94a02ad4ddb9be9c29032'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:09.559164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1104/2207] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:09 2022] DELETE /identity/v3/users/ed1db580dbe94a02ad4ddb9be9c29032 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:09.564030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e0a54-a5a1-456b-8fd0-de002c3f1c5d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:09.564512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e0a54-a5a1-456b-8fd0-de002c3f1c5d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:09.564605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e0a54-a5a1-456b-8fd0-de002c3f1c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:09.584522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-592e0a54-a5a1-456b-8fd0-de002c3f1c5d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:09.585742 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-592e0a54-a5a1-456b-8fd0-de002c3f1c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:09.644413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1104/2208] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:09.653924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:09.655362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:09.713017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70062f-5229-4ecc-90d0-0e4090c6afd8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:09.716701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e7a55-0503-44b0-84e0-5891044e5e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:09.717313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e7a55-0503-44b0-84e0-5891044e5e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:09.717597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e7a55-0503-44b0-84e0-5891044e5e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:09.717875 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e7a55-0503-44b0-84e0-5891044e5e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:09.727611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e7a55-0503-44b0-84e0-5891044e5e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:09.729040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e7a55-0503-44b0-84e0-5891044e5e5b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:09.738565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1105/2209] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:10.010748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e0a54-a5a1-456b-8fd0-de002c3f1c5d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:10.012228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-592e0a54-a5a1-456b-8fd0-de002c3f1c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:10.024617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e0a54-a5a1-456b-8fd0-de002c3f1c5d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:10.029283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:10.030079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:10.030365 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:10.030678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] PATH_INFO: `/v3/projects/14ab99750fff4fd287661c1586894b08` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:10.031853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14ab99750fff4fd287661c1586894b08)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:10.033763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:10.082194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14ab99750fff4fd287661c1586894b08', 'request_id': 'req-e099a370-db80-4c3c-ae83-223dc03ef060'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.089512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14ab99750fff4fd287661c1586894b08', 'request_id': 'req-e099a370-db80-4c3c-ae83-223dc03ef060'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.095481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14ab99750fff4fd287661c1586894b08', 'request_id': 'req-e099a370-db80-4c3c-ae83-223dc03ef060'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.107553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] The token cache is being invalidate because project 14ab99750fff4fd287661c1586894b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:10.107842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.108513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.110257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1105/2210] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:10 2022] DELETE /identity/v3/projects/14ab99750fff4fd287661c1586894b08 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:10.114950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e7a55-0503-44b0-84e0-5891044e5e5b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:10.124958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e7a55-0503-44b0-84e0-5891044e5e5b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:10.185463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:10.186120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:10.186408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:10.186717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] PATH_INFO: `/v3/users/87973478b5124cb9b999c29b3d10c510` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:10.187898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=87973478b5124cb9b999c29b3d10c510)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:10.189872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:10.265232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87973478b5124cb9b999c29b3d10c510'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.276912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87973478b5124cb9b999c29b3d10c510'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.277951 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87973478b5124cb9b999c29b3d10c510'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.278789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87973478b5124cb9b999c29b3d10c510'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.289275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:10.296629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87973478b5124cb9b999c29b3d10c510'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:10.330608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1106/2211] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:10 2022] DELETE /identity/v3/users/87973478b5124cb9b999c29b3d10c510 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:11.201745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:11.205138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:11.250575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099a370-db80-4c3c-ae83-223dc03ef060 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:11.360071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:11.384539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:11.385504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:11.386044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:11.386507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] PATH_INFO: `/v3/projects/fdf2ff1b5d5b42318807917f3f3f8eda` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:11.388692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf2ff1b5d5b42318807917f3f3f8eda)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:11.391168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:11.467511 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf2ff1b5d5b42318807917f3f3f8eda', 'request_id': 'req-f5ba50be-d450-4dd2-a922-4070ef933ed2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.484063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf2ff1b5d5b42318807917f3f3f8eda', 'request_id': 'req-f5ba50be-d450-4dd2-a922-4070ef933ed2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.493533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf2ff1b5d5b42318807917f3f3f8eda', 'request_id': 'req-f5ba50be-d450-4dd2-a922-4070ef933ed2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.517118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] The token cache is being invalidate because project fdf2ff1b5d5b42318807917f3f3f8eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:11.520456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.521525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.530124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1106/2212] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:11 2022] DELETE /identity/v3/projects/fdf2ff1b5d5b42318807917f3f3f8eda => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:11.531399 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:11.533128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:11.558236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808cfbd-f7dd-4b99-b8c1-3a0043fee4eb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:11.747232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:11.748437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:11.750307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:11.750872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] PATH_INFO: `/v3/users/fce97384b9f545ae864fddf2d2a59cb6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:11.753807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce97384b9f545ae864fddf2d2a59cb6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:11.760679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:11.868750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce97384b9f545ae864fddf2d2a59cb6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.880971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce97384b9f545ae864fddf2d2a59cb6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.883807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce97384b9f545ae864fddf2d2a59cb6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.885119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce97384b9f545ae864fddf2d2a59cb6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.905828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73bd3f52-d433-44ce-af18-c57a002ab718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce97384b9f545ae864fddf2d2a59cb6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:11.927244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1107/2213] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:11 2022] DELETE /identity/v3/users/fce97384b9f545ae864fddf2d2a59cb6 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:12.260149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:12.261732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:12.277669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba50be-d450-4dd2-a922-4070ef933ed2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:12.350721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:12.351372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.351649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.351919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] PATH_INFO: `/v3/projects/e54998b0463643e79c4d787c469987c4` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.353087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e54998b0463643e79c4d787c469987c4)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:12.355045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:12.406888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e54998b0463643e79c4d787c469987c4', 'request_id': 'req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.415323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e54998b0463643e79c4d787c469987c4', 'request_id': 'req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.422609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e54998b0463643e79c4d787c469987c4', 'request_id': 'req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.435204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] The token cache is being invalidate because project e54998b0463643e79c4d787c469987c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:12.435487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.436159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-5c894b40-9be9-44f9-bea6-9c9e0d4053b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.437655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1107/2214] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:12 2022] DELETE /identity/v3/projects/e54998b0463643e79c4d787c469987c4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:12.443409 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58028-a60b-4c35-ad38-049a393c2248 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.443701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58028-a60b-4c35-ad38-049a393c2248 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.443980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58028-a60b-4c35-ad38-049a393c2248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.466151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a3f58028-a60b-4c35-ad38-049a393c2248 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:12.467348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3f58028-a60b-4c35-ad38-049a393c2248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:12.546521 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1108/2215] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:12.551919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360e742-3915-445b-95da-bf68ce16a3f4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.552198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360e742-3915-445b-95da-bf68ce16a3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.552468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360e742-3915-445b-95da-bf68ce16a3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.573164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e360e742-3915-445b-95da-bf68ce16a3f4 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:12.574386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e360e742-3915-445b-95da-bf68ce16a3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:12.582400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1108/2216] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:12.586848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f58028-a60b-4c35-ad38-049a393c2248 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:12.591716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:12.598687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f58028-a60b-4c35-ad38-049a393c2248 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:12.604407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acca37-9edb-42d1-b76c-097542562c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:12.605296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acca37-9edb-42d1-b76c-097542562c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.605918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acca37-9edb-42d1-b76c-097542562c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.606201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acca37-9edb-42d1-b76c-097542562c4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.607193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acca37-9edb-42d1-b76c-097542562c4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:12.607503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acca37-9edb-42d1-b76c-097542562c4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:12.609135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acca37-9edb-42d1-b76c-097542562c4d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:12.621939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1109/2217] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:12.626732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360e742-3915-445b-95da-bf68ce16a3f4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:12.633776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:12.639192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360e742-3915-445b-95da-bf68ce16a3f4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:12.645221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a37be-1cdd-4baa-999b-26ff18c2e710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:12.645845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a37be-1cdd-4baa-999b-26ff18c2e710 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.646134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a37be-1cdd-4baa-999b-26ff18c2e710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.646407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a37be-1cdd-4baa-999b-26ff18c2e710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.647121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a37be-1cdd-4baa-999b-26ff18c2e710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:12.649067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a37be-1cdd-4baa-999b-26ff18c2e710 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:12.683663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1109/2218] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:12 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:12.684970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acca37-9edb-42d1-b76c-097542562c4d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:12.695999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acca37-9edb-42d1-b76c-097542562c4d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:12.758368 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2915e0-100e-4121-ac4d-c350570cfd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:12.759602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2915e0-100e-4121-ac4d-c350570cfd6f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.760054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2915e0-100e-4121-ac4d-c350570cfd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.760333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2915e0-100e-4121-ac4d-c350570cfd6f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.761009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2915e0-100e-4121-ac4d-c350570cfd6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:12.762976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2915e0-100e-4121-ac4d-c350570cfd6f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:12.786152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a37be-1cdd-4baa-999b-26ff18c2e710 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:12.788653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5a1a37be-1cdd-4baa-999b-26ff18c2e710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:12.798206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1110/2219] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:12 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:12.803371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2915e0-100e-4121-ac4d-c350570cfd6f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:12.808295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a37be-1cdd-4baa-999b-26ff18c2e710 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:12.814805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:12.815095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2915e0-100e-4121-ac4d-c350570cfd6f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:12.816758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.817202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.817812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] PATH_INFO: `/v3/users/28c687f86caa4cc99565c79e8b2c0aef` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.819153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c687f86caa4cc99565c79e8b2c0aef)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:12.823057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc79f22-8acd-424d-9080-dffc52982a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:12.823057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc79f22-8acd-424d-9080-dffc52982a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.823057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc79f22-8acd-424d-9080-dffc52982a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.823450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:12.823528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc79f22-8acd-424d-9080-dffc52982a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.823528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc79f22-8acd-424d-9080-dffc52982a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:12.824258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc79f22-8acd-424d-9080-dffc52982a49 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:12.838593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1111/2220] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:12.839999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc79f22-8acd-424d-9080-dffc52982a49 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:12.851375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc79f22-8acd-424d-9080-dffc52982a49 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:12.857456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d500d-a1c6-4ef7-a3ad-e9f6568679be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:12.858534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d500d-a1c6-4ef7-a3ad-e9f6568679be admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.859155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d500d-a1c6-4ef7-a3ad-e9f6568679be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.859465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d500d-a1c6-4ef7-a3ad-e9f6568679be admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.860321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d500d-a1c6-4ef7-a3ad-e9f6568679be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:12.862724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d500d-a1c6-4ef7-a3ad-e9f6568679be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:12.874266 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1112/2221] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:12.879449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d500d-a1c6-4ef7-a3ad-e9f6568679be admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:12.881754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c687f86caa4cc99565c79e8b2c0aef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.889575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c687f86caa4cc99565c79e8b2c0aef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.890173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d500d-a1c6-4ef7-a3ad-e9f6568679be admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:12.890611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c687f86caa4cc99565c79e8b2c0aef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.891325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c687f86caa4cc99565c79e8b2c0aef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.895963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a199b-b9c9-4a5e-8aed-34692db5b021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:12.896657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a199b-b9c9-4a5e-8aed-34692db5b021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.896938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a199b-b9c9-4a5e-8aed-34692db5b021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.897221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a199b-b9c9-4a5e-8aed-34692db5b021 admin admin] PATH_INFO: `/v3/projects/847627489c2d483e8594b9bd848f6e95/users/5df6d206b9c14d3d916fe2d3449ed93a/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.906669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1d9ac812-c7d3-4751-87c9-de04465f94db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c687f86caa4cc99565c79e8b2c0aef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:12.915513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a199b-b9c9-4a5e-8aed-34692db5b021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=847627489c2d483e8594b9bd848f6e95, user_id=5df6d206b9c14d3d916fe2d3449ed93a, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:12.918627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a199b-b9c9-4a5e-8aed-34692db5b021 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:12.925155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1110/2222] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:12 2022] DELETE /identity/v3/users/28c687f86caa4cc99565c79e8b2c0aef => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:12.929691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b659c-92f6-4801-8c8f-58cf56566252 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.930177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b659c-92f6-4801-8c8f-58cf56566252 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.930362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b659c-92f6-4801-8c8f-58cf56566252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.938751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1113/2223] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:12 2022] PUT /identity/v3/projects/847627489c2d483e8594b9bd848f6e95/users/5df6d206b9c14d3d916fe2d3449ed93a/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:12.950473 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8812f0-8d6b-497a-a000-1deb2269b7c6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:12.950795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8812f0-8d6b-497a-a000-1deb2269b7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:12.951065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8812f0-8d6b-497a-a000-1deb2269b7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:12.957796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7d4b659c-92f6-4801-8c8f-58cf56566252 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:12.959329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7d4b659c-92f6-4801-8c8f-58cf56566252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:12.984505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9a8812f0-8d6b-497a-a000-1deb2269b7c6 None None] MFA Rules not processed for user `5df6d206b9c14d3d916fe2d3449ed93a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:12.985767 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a8812f0-8d6b-497a-a000-1deb2269b7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:13.075045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1111/2224] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:13.086531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1114/2225] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:13.094875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b659c-92f6-4801-8c8f-58cf56566252 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:13.098581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7d4b659c-92f6-4801-8c8f-58cf56566252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:13.099954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8812f0-8d6b-497a-a000-1deb2269b7c6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:13.101096 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a8812f0-8d6b-497a-a000-1deb2269b7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:13.137625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:13.184481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b659c-92f6-4801-8c8f-58cf56566252 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:13.188725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237254f3-fe79-4f73-a85f-b1c8ed14370c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:13.189359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237254f3-fe79-4f73-a85f-b1c8ed14370c service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:13.189640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237254f3-fe79-4f73-a85f-b1c8ed14370c service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:13.189910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237254f3-fe79-4f73-a85f-b1c8ed14370c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:13.226044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237254f3-fe79-4f73-a85f-b1c8ed14370c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:13.230418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237254f3-fe79-4f73-a85f-b1c8ed14370c service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:13.239866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8812f0-8d6b-497a-a000-1deb2269b7c6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:13.241505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:13.258008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507bf0f-4012-41c7-9228-f41a25f46e6a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '548acf58bf9140ec9f1ba126e10deb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:13.259051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507bf0f-4012-41c7-9228-f41a25f46e6a service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:13.259292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507bf0f-4012-41c7-9228-f41a25f46e6a service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:13.269667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507bf0f-4012-41c7-9228-f41a25f46e6a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:13.281533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1112/2226] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:13.292525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507bf0f-4012-41c7-9228-f41a25f46e6a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:13.294664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507bf0f-4012-41c7-9228-f41a25f46e6a service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:13.308523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1115/2227] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:00:13.525568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b2491-e33a-496f-a9e9-f09c3b59be0f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:13.525847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b2491-e33a-496f-a9e9-f09c3b59be0f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:13.526117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b2491-e33a-496f-a9e9-f09c3b59be0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:13.568014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d29b2491-e33a-496f-a9e9-f09c3b59be0f None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:13.569529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d29b2491-e33a-496f-a9e9-f09c3b59be0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:13.661771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1113/2228] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 05:00:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:13.671253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507bf0f-4012-41c7-9228-f41a25f46e6a service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:13.676939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f507bf0f-4012-41c7-9228-f41a25f46e6a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:13.713177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507bf0f-4012-41c7-9228-f41a25f46e6a service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:13.725123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:13.729198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:13.729547 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:13.729965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] PATH_INFO: `/v3/projects/b1092087b96b4ed39814eeb89da4b0f1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:13.734837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1092087b96b4ed39814eeb89da4b0f1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:13.737696 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:13.755168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b2491-e33a-496f-a9e9-f09c3b59be0f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:13.756809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d29b2491-e33a-496f-a9e9-f09c3b59be0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:13.812604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1092087b96b4ed39814eeb89da4b0f1', 'request_id': 'req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:13.842914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1092087b96b4ed39814eeb89da4b0f1', 'request_id': 'req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:13.859959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1092087b96b4ed39814eeb89da4b0f1', 'request_id': 'req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:13.872515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] The token cache is being invalidate because project b1092087b96b4ed39814eeb89da4b0f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:13.872880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:13.873531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:13.875220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1116/2229] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:13 2022] DELETE /identity/v3/projects/b1092087b96b4ed39814eeb89da4b0f1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:13.880298 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:13.882110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:13.890907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b2491-e33a-496f-a9e9-f09c3b59be0f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:13.977925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971f3a2-5250-4a37-8a5b-ba7a71010ccd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:13.985263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:13.986331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:13.987369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:13.987726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] PATH_INFO: `/v3/users/3712fea816704621bb402b96cb0aceab` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:13.989751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3712fea816704621bb402b96cb0aceab)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:13.992644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:14.007703 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf73f5c-5c5b-471a-a6d7-6ac2caf3b2c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:14.008340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf73f5c-5c5b-471a-a6d7-6ac2caf3b2c5 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:14.008624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf73f5c-5c5b-471a-a6d7-6ac2caf3b2c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:14.008899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf73f5c-5c5b-471a-a6d7-6ac2caf3b2c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:14.011554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cbf73f5c-5c5b-471a-a6d7-6ac2caf3b2c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:14.077732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3712fea816704621bb402b96cb0aceab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.080641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:14.088721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3712fea816704621bb402b96cb0aceab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.089507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3712fea816704621bb402b96cb0aceab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.090236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3712fea816704621bb402b96cb0aceab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.092489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:14.110681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf73f5c-5c5b-471a-a6d7-6ac2caf3b2c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:14.111032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-191b33f3-fdde-44df-8b2e-a12567cbfbe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3712fea816704621bb402b96cb0aceab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.112972 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf73f5c-5c5b-471a-a6d7-6ac2caf3b2c5 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:14.122744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1114/2230] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:00:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 30 05:00:14.126757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1117/2231] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:13 2022] DELETE /identity/v3/users/3712fea816704621bb402b96cb0aceab => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:14.626676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf73f5c-5c5b-471a-a6d7-6ac2caf3b2c5 service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:14.650103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf73f5c-5c5b-471a-a6d7-6ac2caf3b2c5 service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:14.730670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:14.731306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:14.731583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:14.731851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] PATH_INFO: `/v3/projects/774b5e9de7a74c97b52bcaae2d734c03` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:14.733008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774b5e9de7a74c97b52bcaae2d734c03)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:14.734985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:14.788168 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774b5e9de7a74c97b52bcaae2d734c03', 'request_id': 'req-00d99721-b506-4c44-b90e-6abfed6271a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.799475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774b5e9de7a74c97b52bcaae2d734c03', 'request_id': 'req-00d99721-b506-4c44-b90e-6abfed6271a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.805470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774b5e9de7a74c97b52bcaae2d734c03', 'request_id': 'req-00d99721-b506-4c44-b90e-6abfed6271a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.816394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] The token cache is being invalidate because project 774b5e9de7a74c97b52bcaae2d734c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:14.816678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.817372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-00d99721-b506-4c44-b90e-6abfed6271a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:14.819143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1115/2232] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:14 2022] DELETE /identity/v3/projects/774b5e9de7a74c97b52bcaae2d734c03 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:14.825385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c44c4-fece-472f-942d-c7ad7d5902cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:14.825907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c44c4-fece-472f-942d-c7ad7d5902cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:14.826184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c44c4-fece-472f-942d-c7ad7d5902cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:14.856170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-153c44c4-fece-472f-942d-c7ad7d5902cc None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:14.857344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-153c44c4-fece-472f-942d-c7ad7d5902cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:14.885777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2aed-c1d4-4f03-9359-dee8b45fd4a0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:14.886060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2aed-c1d4-4f03-9359-dee8b45fd4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:14.886335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2aed-c1d4-4f03-9359-dee8b45fd4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:14.907582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-53fd2aed-c1d4-4f03-9359-dee8b45fd4a0 None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:14.908868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-53fd2aed-c1d4-4f03-9359-dee8b45fd4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:14.944744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1118/2233] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:14.953273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e2a5-8b08-4190-8c27-dbdd2a2c3913 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:14.953773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e2a5-8b08-4190-8c27-dbdd2a2c3913 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:14.954220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e2a5-8b08-4190-8c27-dbdd2a2c3913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:14.980195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1116/2234] 10.222.0.20 () {58 vars in 967 bytes} [Sat Jul 30 05:00:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:14.981593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1f35e2a5-8b08-4190-8c27-dbdd2a2c3913 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:14.983522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f35e2a5-8b08-4190-8c27-dbdd2a2c3913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:14.997266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd2aed-c1d4-4f03-9359-dee8b45fd4a0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:14.999888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-53fd2aed-c1d4-4f03-9359-dee8b45fd4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:15.003913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1119/2235] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:15.008188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35e2a5-8b08-4190-8c27-dbdd2a2c3913 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.020177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35e2a5-8b08-4190-8c27-dbdd2a2c3913 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.029072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee664162-4fbc-40f7-9f36-40512b611123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.029703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee664162-4fbc-40f7-9f36-40512b611123 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.029984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee664162-4fbc-40f7-9f36-40512b611123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.030416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee664162-4fbc-40f7-9f36-40512b611123 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.031851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee664162-4fbc-40f7-9f36-40512b611123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:15.031946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee664162-4fbc-40f7-9f36-40512b611123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.033572 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee664162-4fbc-40f7-9f36-40512b611123 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.048239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1120/2236] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:15.053793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee664162-4fbc-40f7-9f36-40512b611123 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.064175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:15.070068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee664162-4fbc-40f7-9f36-40512b611123 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.077171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647c01e-636c-49d7-b4cf-9b0a1732ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.077948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647c01e-636c-49d7-b4cf-9b0a1732ad51 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.078281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647c01e-636c-49d7-b4cf-9b0a1732ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.078715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647c01e-636c-49d7-b4cf-9b0a1732ad51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.079470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647c01e-636c-49d7-b4cf-9b0a1732ad51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.083006 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647c01e-636c-49d7-b4cf-9b0a1732ad51 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.126415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1121/2237] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:15 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:15.128378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647c01e-636c-49d7-b4cf-9b0a1732ad51 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.128551 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd2aed-c1d4-4f03-9359-dee8b45fd4a0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.220899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a61561-3918-4ebd-a9f8-ec0b7cf7898d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.221749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a61561-3918-4ebd-a9f8-ec0b7cf7898d service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.222132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a61561-3918-4ebd-a9f8-ec0b7cf7898d service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.222398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a61561-3918-4ebd-a9f8-ec0b7cf7898d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.227552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647c01e-636c-49d7-b4cf-9b0a1732ad51 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.235761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e18241-cc3a-4577-9c64-dd10d1738bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.236401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e18241-cc3a-4577-9c64-dd10d1738bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.236690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e18241-cc3a-4577-9c64-dd10d1738bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.237025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e18241-cc3a-4577-9c64-dd10d1738bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.237703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e18241-cc3a-4577-9c64-dd10d1738bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.239660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e18241-cc3a-4577-9c64-dd10d1738bc5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.275817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1122/2238] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:15 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:15.275817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e18241-cc3a-4577-9c64-dd10d1738bc5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.275817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-77e18241-cc3a-4577-9c64-dd10d1738bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:15.288885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e18241-cc3a-4577-9c64-dd10d1738bc5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.290794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a61561-3918-4ebd-a9f8-ec0b7cf7898d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.292494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a61561-3918-4ebd-a9f8-ec0b7cf7898d service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.294420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.295083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.295365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.295641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] PATH_INFO: `/v3/users/6016a68384714afc9a6eba092a834850` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.296822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6016a68384714afc9a6eba092a834850)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.298796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.301765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1117/2239] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:00:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 30 05:00:15.302590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a61561-3918-4ebd-a9f8-ec0b7cf7898d service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.311852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:15.316771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a61561-3918-4ebd-a9f8-ec0b7cf7898d service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.326726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4dd0a-00b6-4640-917f-c80515df4d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.327895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4dd0a-00b6-4640-917f-c80515df4d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.328216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4dd0a-00b6-4640-917f-c80515df4d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.328370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4dd0a-00b6-4640-917f-c80515df4d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.329105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4dd0a-00b6-4640-917f-c80515df4d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.331325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4dd0a-00b6-4640-917f-c80515df4d54 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.342873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1118/2240] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:15.347411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4dd0a-00b6-4640-917f-c80515df4d54 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.357586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4dd0a-00b6-4640-917f-c80515df4d54 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.361690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6016a68384714afc9a6eba092a834850'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:15.366983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f412c80-e5f1-40c3-bcfa-76f27c91979a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.371518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f412c80-e5f1-40c3-bcfa-76f27c91979a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.372154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f412c80-e5f1-40c3-bcfa-76f27c91979a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.372492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f412c80-e5f1-40c3-bcfa-76f27c91979a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.372658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6016a68384714afc9a6eba092a834850'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:15.373350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f412c80-e5f1-40c3-bcfa-76f27c91979a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.373467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6016a68384714afc9a6eba092a834850'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:15.374327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6016a68384714afc9a6eba092a834850'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:15.375402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f412c80-e5f1-40c3-bcfa-76f27c91979a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.390309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c3aa2e1-7182-4141-9dd5-34bbc8ca3f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6016a68384714afc9a6eba092a834850'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:15.390507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1119/2241] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:15.396509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f412c80-e5f1-40c3-bcfa-76f27c91979a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.398930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4f412c80-e5f1-40c3-bcfa-76f27c91979a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:15.408487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1123/2242] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:15 2022] DELETE /identity/v3/users/6016a68384714afc9a6eba092a834850 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:15.416261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62221b05-21a2-495a-a162-1ad6ad03c6a5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.416928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62221b05-21a2-495a-a162-1ad6ad03c6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.417369 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62221b05-21a2-495a-a162-1ad6ad03c6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.425826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f412c80-e5f1-40c3-bcfa-76f27c91979a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.442338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-62221b05-21a2-495a-a162-1ad6ad03c6a5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:15.443886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-62221b05-21a2-495a-a162-1ad6ad03c6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:15.475766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:15.504542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8fcf5-ede2-4844-b1ab-4dfdf7b91850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.505239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8fcf5-ede2-4844-b1ab-4dfdf7b91850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.505982 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8fcf5-ede2-4844-b1ab-4dfdf7b91850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.506271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8fcf5-ede2-4844-b1ab-4dfdf7b91850 admin admin] PATH_INFO: `/v3/projects/fe31334f5f4540c68b911f4871c3cc77/users/efeb4d9493554d2fb19b0409b241fbf6/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.530564 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1124/2243] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:15.536965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8fcf5-ede2-4844-b1ab-4dfdf7b91850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe31334f5f4540c68b911f4871c3cc77, user_id=efeb4d9493554d2fb19b0409b241fbf6, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.539757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8fcf5-ede2-4844-b1ab-4dfdf7b91850 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.541824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62221b05-21a2-495a-a162-1ad6ad03c6a5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.543859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-62221b05-21a2-495a-a162-1ad6ad03c6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:15.559700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1120/2244] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:15 2022] PUT /identity/v3/projects/fe31334f5f4540c68b911f4871c3cc77/users/efeb4d9493554d2fb19b0409b241fbf6/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:15.608657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b6b0-e6bc-4aac-b808-a74a383bfefc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.608984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b6b0-e6bc-4aac-b808-a74a383bfefc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.610181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b6b0-e6bc-4aac-b808-a74a383bfefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.648695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0af2b6b0-e6bc-4aac-b808-a74a383bfefc None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:15.650708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0af2b6b0-e6bc-4aac-b808-a74a383bfefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:15.665584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62221b05-21a2-495a-a162-1ad6ad03c6a5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.729843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:15.733021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1121/2245] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 05:00:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:15.734703 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80076a4-7b10-4666-a4a3-4156d46f0305 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.735198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80076a4-7b10-4666-a4a3-4156d46f0305 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.735672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80076a4-7b10-4666-a4a3-4156d46f0305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.739125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd245b-b44e-4b39-ae4b-d058f5c9c551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.740128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd245b-b44e-4b39-ae4b-d058f5c9c551 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.740439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd245b-b44e-4b39-ae4b-d058f5c9c551 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.740780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd245b-b44e-4b39-ae4b-d058f5c9c551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.774807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f80076a4-7b10-4666-a4a3-4156d46f0305 None None] MFA Rules not processed for user `efeb4d9493554d2fb19b0409b241fbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:15.776375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f80076a4-7b10-4666-a4a3-4156d46f0305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:15.826746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd245b-b44e-4b39-ae4b-d058f5c9c551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.828221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd245b-b44e-4b39-ae4b-d058f5c9c551 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.838103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1125/2246] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:15.843374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd245b-b44e-4b39-ae4b-d058f5c9c551 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.844788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fddd245b-b44e-4b39-ae4b-d058f5c9c551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:15.855291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1122/2247] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:15.868104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80076a4-7b10-4666-a4a3-4156d46f0305 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:15.869497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f80076a4-7b10-4666-a4a3-4156d46f0305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:15.930268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd245b-b44e-4b39-ae4b-d058f5c9c551 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.936422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80076a4-7b10-4666-a4a3-4156d46f0305 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:15.936925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f8599-c529-4057-82b1-f07f6c00ccc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.937565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f8599-c529-4057-82b1-f07f6c00ccc1 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.937865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f8599-c529-4057-82b1-f07f6c00ccc1 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.938145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f8599-c529-4057-82b1-f07f6c00ccc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.941113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b301fa-d9da-42b4-b6bf-b7a4e092430d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:15.941813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b301fa-d9da-42b4-b6bf-b7a4e092430d service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:15.942199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b301fa-d9da-42b4-b6bf-b7a4e092430d service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:15.942569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b301fa-d9da-42b4-b6bf-b7a4e092430d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:15.955738 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f8599-c529-4057-82b1-f07f6c00ccc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.957494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f8599-c529-4057-82b1-f07f6c00ccc1 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.965284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b301fa-d9da-42b4-b6bf-b7a4e092430d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:15.966783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b301fa-d9da-42b4-b6bf-b7a4e092430d service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:15.975334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1126/2248] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:00:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 05:00:15.990711 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1123/2249] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:00:16.231680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f8599-c529-4057-82b1-f07f6c00ccc1 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:16.233816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f0f8599-c529-4057-82b1-f07f6c00ccc1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:16.281567 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f8599-c529-4057-82b1-f07f6c00ccc1 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:16.290503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:16.291281 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:16.291567 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:16.291843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] PATH_INFO: `/v3/projects/497d9d6d607c48fab65b3604b24806af` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:16.293477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=497d9d6d607c48fab65b3604b24806af)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:16.295777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:16.400996 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '497d9d6d607c48fab65b3604b24806af', 'request_id': 'req-5ca978a2-93e9-4c27-a837-f0a0253f3307'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.425625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '497d9d6d607c48fab65b3604b24806af', 'request_id': 'req-5ca978a2-93e9-4c27-a837-f0a0253f3307'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.439199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '497d9d6d607c48fab65b3604b24806af', 'request_id': 'req-5ca978a2-93e9-4c27-a837-f0a0253f3307'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.457496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] The token cache is being invalidate because project 497d9d6d607c48fab65b3604b24806af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:16.457886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.461514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ca978a2-93e9-4c27-a837-f0a0253f3307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.461514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1127/2250] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:16 2022] DELETE /identity/v3/projects/497d9d6d607c48fab65b3604b24806af => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:16.464892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b301fa-d9da-42b4-b6bf-b7a4e092430d service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:16.476752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:16.482906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b301fa-d9da-42b4-b6bf-b7a4e092430d service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:16.499170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c566f-f59f-45a1-b4ed-1e1507c4c4fe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:16.499627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c566f-f59f-45a1-b4ed-1e1507c4c4fe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:16.499740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c566f-f59f-45a1-b4ed-1e1507c4c4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:16.514355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:16.522154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-457c566f-f59f-45a1-b4ed-1e1507c4c4fe None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:16.524049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-457c566f-f59f-45a1-b4ed-1e1507c4c4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:16.586880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:16.587782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:16.588071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:16.588347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] PATH_INFO: `/v3/users/fd01d1225b134c75b3f69f18a3999037` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:16.589545 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd01d1225b134c75b3f69f18a3999037)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:16.591514 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:16.592158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1128/2251] 10.222.0.20 () {58 vars in 967 bytes} [Sat Jul 30 05:00:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:16.606071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c566f-f59f-45a1-b4ed-1e1507c4c4fe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:16.607588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-457c566f-f59f-45a1-b4ed-1e1507c4c4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:16.651348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd01d1225b134c75b3f69f18a3999037'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.661910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd01d1225b134c75b3f69f18a3999037'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.663345 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd01d1225b134c75b3f69f18a3999037'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.664091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd01d1225b134c75b3f69f18a3999037'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.681213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd01d1225b134c75b3f69f18a3999037'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:16.688092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c566f-f59f-45a1-b4ed-1e1507c4c4fe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:16.696921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2bf88-679d-460d-92f5-c389afe0f51b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:16.697927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2bf88-679d-460d-92f5-c389afe0f51b service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:16.698201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2bf88-679d-460d-92f5-c389afe0f51b service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:16.698508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2bf88-679d-460d-92f5-c389afe0f51b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:16.701666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c8c2bf88-679d-460d-92f5-c389afe0f51b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:16.710220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1124/2252] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:16 2022] DELETE /identity/v3/users/fd01d1225b134c75b3f69f18a3999037 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:16.714548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2bf88-679d-460d-92f5-c389afe0f51b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:16.716169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2bf88-679d-460d-92f5-c389afe0f51b service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:16.725627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1129/2253] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:00:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 30 05:00:17.268050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:17.268050 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:17.351245 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51479c-4e09-41df-a00c-b06d2bfca172 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:17.359968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:17.360647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:17.362668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:17.363482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] PATH_INFO: `/v3/projects/9f3dad2790714286bbc7b157afb2e65c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:17.364973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f3dad2790714286bbc7b157afb2e65c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:17.367563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:17.407201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:17.431059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f3dad2790714286bbc7b157afb2e65c', 'request_id': 'req-a38aebc5-b54c-4fea-b58b-eb65e933d698'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:17.441754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f3dad2790714286bbc7b157afb2e65c', 'request_id': 'req-a38aebc5-b54c-4fea-b58b-eb65e933d698'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:17.452279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f3dad2790714286bbc7b157afb2e65c', 'request_id': 'req-a38aebc5-b54c-4fea-b58b-eb65e933d698'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:17.464782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] The token cache is being invalidate because project 9f3dad2790714286bbc7b157afb2e65c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:17.465074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:17.465746 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a38aebc5-b54c-4fea-b58b-eb65e933d698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:17.467312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1125/2254] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:17 2022] DELETE /identity/v3/projects/9f3dad2790714286bbc7b157afb2e65c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:17.473796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11462240-65cd-4032-a35a-8ed881e5a18e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:17.474280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11462240-65cd-4032-a35a-8ed881e5a18e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:17.474493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11462240-65cd-4032-a35a-8ed881e5a18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:17.518920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-11462240-65cd-4032-a35a-8ed881e5a18e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:17.520407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11462240-65cd-4032-a35a-8ed881e5a18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:17.598210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1130/2255] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:17.605902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b748c3-56da-4034-b7ce-8769a5aa0343 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:17.606198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b748c3-56da-4034-b7ce-8769a5aa0343 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:17.606618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b748c3-56da-4034-b7ce-8769a5aa0343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:17.630628 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f3b748c3-56da-4034-b7ce-8769a5aa0343 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:17.631858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f3b748c3-56da-4034-b7ce-8769a5aa0343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:17.647232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1126/2256] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:17.648985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11462240-65cd-4032-a35a-8ed881e5a18e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:17.669056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11462240-65cd-4032-a35a-8ed881e5a18e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:17.679250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abdca8-924b-4653-99c7-2dfc9608ed02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:17.680976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abdca8-924b-4653-99c7-2dfc9608ed02 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:17.680976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abdca8-924b-4653-99c7-2dfc9608ed02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:17.680976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abdca8-924b-4653-99c7-2dfc9608ed02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:17.681225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abdca8-924b-4653-99c7-2dfc9608ed02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:17.681405 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abdca8-924b-4653-99c7-2dfc9608ed02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:17.683044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abdca8-924b-4653-99c7-2dfc9608ed02 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:17.697921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1131/2257] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:17.702682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b748c3-56da-4034-b7ce-8769a5aa0343 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:17.713282 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b748c3-56da-4034-b7ce-8769a5aa0343 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:17.722826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6514673-ff25-471f-b707-335f5c73931c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:17.723778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6514673-ff25-471f-b707-335f5c73931c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:17.724257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6514673-ff25-471f-b707-335f5c73931c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:17.724724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6514673-ff25-471f-b707-335f5c73931c admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:17.725603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6514673-ff25-471f-b707-335f5c73931c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:17.727791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6514673-ff25-471f-b707-335f5c73931c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:17.760846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1127/2258] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:17 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:17.763888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abdca8-924b-4653-99c7-2dfc9608ed02 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:17.775551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abdca8-924b-4653-99c7-2dfc9608ed02 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:17.811161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:17.847109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f327d59-47bb-468e-8f59-d6c76631fe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:17.847981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f327d59-47bb-468e-8f59-d6c76631fe55 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:17.848280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f327d59-47bb-468e-8f59-d6c76631fe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:17.848558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f327d59-47bb-468e-8f59-d6c76631fe55 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:17.849229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f327d59-47bb-468e-8f59-d6c76631fe55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:17.851199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f327d59-47bb-468e-8f59-d6c76631fe55 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:17.882067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1132/2259] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:17 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:17.886123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6514673-ff25-471f-b707-335f5c73931c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:17.898340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6514673-ff25-471f-b707-335f5c73931c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:17.903423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b02c30-ea02-42fd-9949-e43737aac6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:17.904041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b02c30-ea02-42fd-9949-e43737aac6fe admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:17.904320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b02c30-ea02-42fd-9949-e43737aac6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:17.904651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b02c30-ea02-42fd-9949-e43737aac6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:17.905278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b02c30-ea02-42fd-9949-e43737aac6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:17.906920 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b02c30-ea02-42fd-9949-e43737aac6fe admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:17.918283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1128/2260] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:17.922840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f327d59-47bb-468e-8f59-d6c76631fe55 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:17.934288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f327d59-47bb-468e-8f59-d6c76631fe55 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:17.939902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644833d-903c-45aa-93f2-ff69990f18a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:17.940733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644833d-903c-45aa-93f2-ff69990f18a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:17.941021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644833d-903c-45aa-93f2-ff69990f18a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:17.941292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644833d-903c-45aa-93f2-ff69990f18a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:17.942005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644833d-903c-45aa-93f2-ff69990f18a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:17.943648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644833d-903c-45aa-93f2-ff69990f18a1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:17.958559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1133/2261] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:17.963535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b02c30-ea02-42fd-9949-e43737aac6fe admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:17.975595 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b02c30-ea02-42fd-9949-e43737aac6fe admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:17.980853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b502837-b6aa-493a-b9e6-aa9d1d25cf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:17.981580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b502837-b6aa-493a-b9e6-aa9d1d25cf42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:17.981869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b502837-b6aa-493a-b9e6-aa9d1d25cf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:17.982122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b502837-b6aa-493a-b9e6-aa9d1d25cf42 admin admin] PATH_INFO: `/v3/projects/c5e89b423ba143349529c28d9215bfa8/users/175a811c9e8b428491e170185c26f0cd/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.008782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b502837-b6aa-493a-b9e6-aa9d1d25cf42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e89b423ba143349529c28d9215bfa8, user_id=175a811c9e8b428491e170185c26f0cd, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.012602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b502837-b6aa-493a-b9e6-aa9d1d25cf42 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.028639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1129/2262] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:17 2022] PUT /identity/v3/projects/c5e89b423ba143349529c28d9215bfa8/users/175a811c9e8b428491e170185c26f0cd/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:18.040282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76a532-e88e-4d8d-8dbe-aafdefcb56d6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.040566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76a532-e88e-4d8d-8dbe-aafdefcb56d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.040841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76a532-e88e-4d8d-8dbe-aafdefcb56d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.068410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4c76a532-e88e-4d8d-8dbe-aafdefcb56d6 None None] MFA Rules not processed for user `175a811c9e8b428491e170185c26f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:18.069603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4c76a532-e88e-4d8d-8dbe-aafdefcb56d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:18.153195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1134/2263] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:18.158183 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b502837-b6aa-493a-b9e6-aa9d1d25cf42 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:18.258684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b502837-b6aa-493a-b9e6-aa9d1d25cf42 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:18.265020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae0dcd-dd12-48bf-b209-0552916e685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:18.265648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae0dcd-dd12-48bf-b209-0552916e685c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.265924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae0dcd-dd12-48bf-b209-0552916e685c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.266191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae0dcd-dd12-48bf-b209-0552916e685c admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.267138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae0dcd-dd12-48bf-b209-0552916e685c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.269528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae0dcd-dd12-48bf-b209-0552916e685c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.302477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1130/2264] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:18 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:18.305996 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76a532-e88e-4d8d-8dbe-aafdefcb56d6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:18.318842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76a532-e88e-4d8d-8dbe-aafdefcb56d6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:18.379768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3cfb2-cd6e-461a-9ba5-20398899e96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:18.380468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3cfb2-cd6e-461a-9ba5-20398899e96b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.380783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3cfb2-cd6e-461a-9ba5-20398899e96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.381059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3cfb2-cd6e-461a-9ba5-20398899e96b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.381731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3cfb2-cd6e-461a-9ba5-20398899e96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.383742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3cfb2-cd6e-461a-9ba5-20398899e96b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.410489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1135/2265] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:18 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:18.412295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae0dcd-dd12-48bf-b209-0552916e685c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:18.423093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae0dcd-dd12-48bf-b209-0552916e685c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:18.427934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b88511-d659-4814-82d6-670ac1cc717c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:18.428573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b88511-d659-4814-82d6-670ac1cc717c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.428853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b88511-d659-4814-82d6-670ac1cc717c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.429130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b88511-d659-4814-82d6-670ac1cc717c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.429831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b88511-d659-4814-82d6-670ac1cc717c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.431476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b88511-d659-4814-82d6-670ac1cc717c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.442539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1131/2266] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:18.446610 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3cfb2-cd6e-461a-9ba5-20398899e96b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:18.457577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3cfb2-cd6e-461a-9ba5-20398899e96b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:18.462536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d11ec-d218-45ec-924c-6fc2c7ec4d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:18.463215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d11ec-d218-45ec-924c-6fc2c7ec4d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.463497 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d11ec-d218-45ec-924c-6fc2c7ec4d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.463772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d11ec-d218-45ec-924c-6fc2c7ec4d6b admin admin] PATH_INFO: `/v3/projects/f5c82267f9224545a52eab89d60dd07e/users/101c113ce18a4d75beb7887997fc4af9/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.481158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d11ec-d218-45ec-924c-6fc2c7ec4d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c82267f9224545a52eab89d60dd07e, user_id=101c113ce18a4d75beb7887997fc4af9, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.483936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d11ec-d218-45ec-924c-6fc2c7ec4d6b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.497309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1136/2267] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:18 2022] PUT /identity/v3/projects/f5c82267f9224545a52eab89d60dd07e/users/101c113ce18a4d75beb7887997fc4af9/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:18.501351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b88511-d659-4814-82d6-670ac1cc717c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:18.512065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b88511-d659-4814-82d6-670ac1cc717c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:18.532145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:18.573112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32723030-42ee-48fd-9350-cea14629316c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:18.573740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723030-42ee-48fd-9350-cea14629316c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.574019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723030-42ee-48fd-9350-cea14629316c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.574289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32723030-42ee-48fd-9350-cea14629316c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.575027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32723030-42ee-48fd-9350-cea14629316c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.576658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32723030-42ee-48fd-9350-cea14629316c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.587125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1132/2268] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:18.591145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d11ec-d218-45ec-924c-6fc2c7ec4d6b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:18.596566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:18.602878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d11ec-d218-45ec-924c-6fc2c7ec4d6b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:18.607935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf5f87-8d42-4829-9e3b-d47392014021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:18.608591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf5f87-8d42-4829-9e3b-d47392014021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.608873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf5f87-8d42-4829-9e3b-d47392014021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.609146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf5f87-8d42-4829-9e3b-d47392014021 admin admin] PATH_INFO: `/v3/projects/f5c82267f9224545a52eab89d60dd07e/users/101c113ce18a4d75beb7887997fc4af9/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.611139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf5f87-8d42-4829-9e3b-d47392014021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c82267f9224545a52eab89d60dd07e, user_id=101c113ce18a4d75beb7887997fc4af9, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.614274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf5f87-8d42-4829-9e3b-d47392014021 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.624367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1137/2269] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:18 2022] PUT /identity/v3/projects/f5c82267f9224545a52eab89d60dd07e/users/101c113ce18a4d75beb7887997fc4af9/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:18.636681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fad64-9ae9-4fec-ac09-245c29f6c51f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.636967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fad64-9ae9-4fec-ac09-245c29f6c51f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.637246 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fad64-9ae9-4fec-ac09-245c29f6c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.665583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f50fad64-9ae9-4fec-ac09-245c29f6c51f None None] MFA Rules not processed for user `101c113ce18a4d75beb7887997fc4af9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:18.666873 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f50fad64-9ae9-4fec-ac09-245c29f6c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:18.739583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1133/2270] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:18.751116 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf5f87-8d42-4829-9e3b-d47392014021 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:18.753671 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b1cf5f87-8d42-4829-9e3b-d47392014021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:18.757070 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fad64-9ae9-4fec-ac09-245c29f6c51f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:18.758180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f50fad64-9ae9-4fec-ac09-245c29f6c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:18.820478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf5f87-8d42-4829-9e3b-d47392014021 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:18.824136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018db56-a8c2-4881-8fa5-443d7b34fb1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:18.824750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018db56-a8c2-4881-8fa5-443d7b34fb1b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.825024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018db56-a8c2-4881-8fa5-443d7b34fb1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.825294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018db56-a8c2-4881-8fa5-443d7b34fb1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.827049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fad64-9ae9-4fec-ac09-245c29f6c51f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:18.832408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:18.833194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.833594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.834179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] PATH_INFO: `/v3/users/efeb4d9493554d2fb19b0409b241fbf6` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.835293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018db56-a8c2-4881-8fa5-443d7b34fb1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.836164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efeb4d9493554d2fb19b0409b241fbf6)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.836731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018db56-a8c2-4881-8fa5-443d7b34fb1b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.839475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.848502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1138/2271] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:18.882220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018db56-a8c2-4881-8fa5-443d7b34fb1b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:18.893501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018db56-a8c2-4881-8fa5-443d7b34fb1b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:18.894601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efeb4d9493554d2fb19b0409b241fbf6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:18.895986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:18.899478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:18.900325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.900645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.900934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] PATH_INFO: `/v3/users/175a811c9e8b428491e170185c26f0cd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.902510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=175a811c9e8b428491e170185c26f0cd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:18.904471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:18.904746 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efeb4d9493554d2fb19b0409b241fbf6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:18.905432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efeb4d9493554d2fb19b0409b241fbf6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:18.906053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efeb4d9493554d2fb19b0409b241fbf6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:18.920354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d169e1fe-0768-4415-bec0-af170c7d5e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efeb4d9493554d2fb19b0409b241fbf6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:18.946425 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1134/2272] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:18 2022] DELETE /identity/v3/users/efeb4d9493554d2fb19b0409b241fbf6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:18.950809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a8ae2-2001-43e1-a972-acb8dca9cf35 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:18.951162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a8ae2-2001-43e1-a972-acb8dca9cf35 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:18.951501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a8ae2-2001-43e1-a972-acb8dca9cf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:18.966130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175a811c9e8b428491e170185c26f0cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:18.970228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-591a8ae2-2001-43e1-a972-acb8dca9cf35 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:18.972038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-591a8ae2-2001-43e1-a972-acb8dca9cf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:18.974090 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175a811c9e8b428491e170185c26f0cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:18.974795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175a811c9e8b428491e170185c26f0cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:18.975433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175a811c9e8b428491e170185c26f0cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:18.988030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f9d24a89-a436-4605-b89c-11ae64c93487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175a811c9e8b428491e170185c26f0cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.002193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1139/2273] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:18 2022] DELETE /identity/v3/users/175a811c9e8b428491e170185c26f0cd => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:19.008908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a8541-df77-44ef-bf07-7f61b5347f9c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:19.009447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a8541-df77-44ef-bf07-7f61b5347f9c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:19.009942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a8541-df77-44ef-bf07-7f61b5347f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:19.031804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d11a8541-df77-44ef-bf07-7f61b5347f9c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:19.033013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d11a8541-df77-44ef-bf07-7f61b5347f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.047554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1135/2274] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:19.049401 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a8ae2-2001-43e1-a972-acb8dca9cf35 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:19.051664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-591a8ae2-2001-43e1-a972-acb8dca9cf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.063869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a8ae2-2001-43e1-a972-acb8dca9cf35 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:19.096164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1140/2275] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:19.097745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a8541-df77-44ef-bf07-7f61b5347f9c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:19.099065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d11a8541-df77-44ef-bf07-7f61b5347f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.105700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:19.129364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:19.130006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:19.130280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:19.130589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] PATH_INFO: `/v3/users/929d7587f49445ed9c86d53a1d888648` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:19.131777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=929d7587f49445ed9c86d53a1d888648)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:19.133688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:19.167200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a8541-df77-44ef-bf07-7f61b5347f9c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:19.171483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423e04c-7693-4fd9-9511-741ac451ba30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:19.172313 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423e04c-7693-4fd9-9511-741ac451ba30 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:19.172604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423e04c-7693-4fd9-9511-741ac451ba30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:19.172885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423e04c-7693-4fd9-9511-741ac451ba30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:19.183102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '929d7587f49445ed9c86d53a1d888648'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.183267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423e04c-7693-4fd9-9511-741ac451ba30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:19.184553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423e04c-7693-4fd9-9511-741ac451ba30 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:19.190870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '929d7587f49445ed9c86d53a1d888648'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.191603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '929d7587f49445ed9c86d53a1d888648'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.192440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '929d7587f49445ed9c86d53a1d888648'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.196139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1141/2276] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:19.197014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423e04c-7693-4fd9-9511-741ac451ba30 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:19.198689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0423e04c-7693-4fd9-9511-741ac451ba30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.202747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:19.205188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f38bc373-4239-4244-ad35-e1f3b311c570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '929d7587f49445ed9c86d53a1d888648'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.211504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423e04c-7693-4fd9-9511-741ac451ba30 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:19.216065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:19.216789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:19.217131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:19.217422 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:19.219692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.226042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1136/2277] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:19 2022] DELETE /identity/v3/users/929d7587f49445ed9c86d53a1d888648 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:19.230502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b25d7-67c4-440b-ac4c-e6b20a6e654e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:19.231040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b25d7-67c4-440b-ac4c-e6b20a6e654e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:19.231381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b25d7-67c4-440b-ac4c-e6b20a6e654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:19.233587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:19.235565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:19.249644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1142/2278] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:19.261805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-147b25d7-67c4-440b-ac4c-e6b20a6e654e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:19.263745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-147b25d7-67c4-440b-ac4c-e6b20a6e654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.397199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1137/2279] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:19.424533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:19.426344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.440583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54475a3a-64f8-4104-9c18-342810414fcb service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:19.531990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67419c-df60-4565-93a5-ee6bb9f9d307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:19.532641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67419c-df60-4565-93a5-ee6bb9f9d307 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:19.532886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67419c-df60-4565-93a5-ee6bb9f9d307 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:19.533162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67419c-df60-4565-93a5-ee6bb9f9d307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:19.543748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67419c-df60-4565-93a5-ee6bb9f9d307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:19.545217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67419c-df60-4565-93a5-ee6bb9f9d307 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:19.566026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1143/2280] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:19.736300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b25d7-67c4-440b-ac4c-e6b20a6e654e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:19.739159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-147b25d7-67c4-440b-ac4c-e6b20a6e654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.745508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67419c-df60-4565-93a5-ee6bb9f9d307 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:19.747078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b67419c-df60-4565-93a5-ee6bb9f9d307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.766617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b25d7-67c4-440b-ac4c-e6b20a6e654e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:19.780113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67419c-df60-4565-93a5-ee6bb9f9d307 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:19.787370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:19.788049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:19.788325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:19.788597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] PATH_INFO: `/v3/projects/c5e89b423ba143349529c28d9215bfa8` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:19.790052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e89b423ba143349529c28d9215bfa8)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:19.791203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:19.792156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:19.792295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:19.792381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:19.792632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] PATH_INFO: `/v3/projects/fe31334f5f4540c68b911f4871c3cc77` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:19.794287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe31334f5f4540c68b911f4871c3cc77)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:19.804719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:19.858807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e89b423ba143349529c28d9215bfa8', 'request_id': 'req-6789295f-2ef0-4c7c-9ef3-3963e2224465'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.867271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e89b423ba143349529c28d9215bfa8', 'request_id': 'req-6789295f-2ef0-4c7c-9ef3-3963e2224465'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.874275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e89b423ba143349529c28d9215bfa8', 'request_id': 'req-6789295f-2ef0-4c7c-9ef3-3963e2224465'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.876659 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe31334f5f4540c68b911f4871c3cc77', 'request_id': 'req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.885807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe31334f5f4540c68b911f4871c3cc77', 'request_id': 'req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.887111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] The token cache is being invalidate because project c5e89b423ba143349529c28d9215bfa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:19.887589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.888254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.889779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1138/2281] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:19 2022] DELETE /identity/v3/projects/c5e89b423ba143349529c28d9215bfa8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:19.894615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:19.894752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe31334f5f4540c68b911f4871c3cc77', 'request_id': 'req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.895997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.905834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] The token cache is being invalidate because project fe31334f5f4540c68b911f4871c3cc77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:19.906123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.906779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:19.908282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1144/2282] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:19 2022] DELETE /identity/v3/projects/fe31334f5f4540c68b911f4871c3cc77 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:19.909771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:19.911071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:19.944927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:19.969548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789295f-2ef0-4c7c-9ef3-3963e2224465 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:19.982716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:19.993584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0994a0-6c4b-4ae8-aafd-8ca8aca03802 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:19.998563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:19.999217 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:19.999560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.000144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] PATH_INFO: `/v3/users/101c113ce18a4d75beb7887997fc4af9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.001086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=101c113ce18a4d75beb7887997fc4af9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.009008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.040901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.041569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.041850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.042125 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] PATH_INFO: `/v3/projects/5dc01f77fe9745369cef62c8379f9209` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.043678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc01f77fe9745369cef62c8379f9209)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.045997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.063131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101c113ce18a4d75beb7887997fc4af9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.071418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101c113ce18a4d75beb7887997fc4af9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.072144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101c113ce18a4d75beb7887997fc4af9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.072790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101c113ce18a4d75beb7887997fc4af9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.086004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1b303c5d-57ac-419c-839c-d855605d24eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101c113ce18a4d75beb7887997fc4af9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.095015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc01f77fe9745369cef62c8379f9209', 'request_id': 'req-67238064-ae35-455d-a6f4-d3341f43b8fe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.100081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1145/2283] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:19 2022] DELETE /identity/v3/users/101c113ce18a4d75beb7887997fc4af9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:20.101557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca6108-e512-4191-a332-e147d2125638 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.101934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca6108-e512-4191-a332-e147d2125638 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.102125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca6108-e512-4191-a332-e147d2125638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.102541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc01f77fe9745369cef62c8379f9209', 'request_id': 'req-67238064-ae35-455d-a6f4-d3341f43b8fe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.108741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc01f77fe9745369cef62c8379f9209', 'request_id': 'req-67238064-ae35-455d-a6f4-d3341f43b8fe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.122413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] The token cache is being invalidate because project 5dc01f77fe9745369cef62c8379f9209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:20.122731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d1ca6108-e512-4191-a332-e147d2125638 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:20.122848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.123535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-67238064-ae35-455d-a6f4-d3341f43b8fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.123906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1ca6108-e512-4191-a332-e147d2125638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:20.125367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1139/2284] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:19 2022] DELETE /identity/v3/projects/5dc01f77fe9745369cef62c8379f9209 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:20.132422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56099dfc-6ae2-42df-8812-23bca2678b79 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.132779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56099dfc-6ae2-42df-8812-23bca2678b79 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.133131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56099dfc-6ae2-42df-8812-23bca2678b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.154300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-56099dfc-6ae2-42df-8812-23bca2678b79 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:20.155730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-56099dfc-6ae2-42df-8812-23bca2678b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:20.203605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1146/2285] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:20.214465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547753a2-8186-4bd9-b73c-19a3b763125f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.214465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547753a2-8186-4bd9-b73c-19a3b763125f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.214465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547753a2-8186-4bd9-b73c-19a3b763125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.235865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-547753a2-8186-4bd9-b73c-19a3b763125f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:20.237967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-547753a2-8186-4bd9-b73c-19a3b763125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:20.239386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1140/2286] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:20.246600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822324c-d328-454c-991a-a1b572f31cc2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.246888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822324c-d328-454c-991a-a1b572f31cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.247165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822324c-d328-454c-991a-a1b572f31cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.255057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1147/2287] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:20.256005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547753a2-8186-4bd9-b73c-19a3b763125f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.266919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547753a2-8186-4bd9-b73c-19a3b763125f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.273394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6822324c-d328-454c-991a-a1b572f31cc2 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:20.273534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf80ca0-0a8e-41d4-9d02-660602a4f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.274039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf80ca0-0a8e-41d4-9d02-660602a4f36b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.274365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf80ca0-0a8e-41d4-9d02-660602a4f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.274660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf80ca0-0a8e-41d4-9d02-660602a4f36b admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.274778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6822324c-d328-454c-991a-a1b572f31cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:20.275551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf80ca0-0a8e-41d4-9d02-660602a4f36b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:20.275782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf80ca0-0a8e-41d4-9d02-660602a4f36b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.277529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf80ca0-0a8e-41d4-9d02-660602a4f36b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.294835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1141/2288] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:20.295062 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1148/2289] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:20.296188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf80ca0-0a8e-41d4-9d02-660602a4f36b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.296300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822324c-d328-454c-991a-a1b572f31cc2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.309967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822324c-d328-454c-991a-a1b572f31cc2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.314059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf80ca0-0a8e-41d4-9d02-660602a4f36b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.317614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdedb7-a905-4841-b68c-a456ae593406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.318245 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdedb7-a905-4841-b68c-a456ae593406 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.318775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdedb7-a905-4841-b68c-a456ae593406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.319193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdedb7-a905-4841-b68c-a456ae593406 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.320237 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b2821-81cc-4ab7-852a-56a5d68f5765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.320537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcdedb7-a905-4841-b68c-a456ae593406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:20.320841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcdedb7-a905-4841-b68c-a456ae593406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.320971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b2821-81cc-4ab7-852a-56a5d68f5765 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.321218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b2821-81cc-4ab7-852a-56a5d68f5765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.321500 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b2821-81cc-4ab7-852a-56a5d68f5765 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.322183 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b2821-81cc-4ab7-852a-56a5d68f5765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.322472 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcdedb7-a905-4841-b68c-a456ae593406 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.324117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b2821-81cc-4ab7-852a-56a5d68f5765 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.338494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1142/2290] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:20.341528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdedb7-a905-4841-b68c-a456ae593406 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.351326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:20.356353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdedb7-a905-4841-b68c-a456ae593406 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.364838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1149/2291] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/projects => generated 397 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:20.366941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db4a9b-0914-4c77-b645-f568f9460cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.368275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db4a9b-0914-4c77-b645-f568f9460cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.368553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db4a9b-0914-4c77-b645-f568f9460cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.368823 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db4a9b-0914-4c77-b645-f568f9460cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.368964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b2821-81cc-4ab7-852a-56a5d68f5765 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.369495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db4a9b-0914-4c77-b645-f568f9460cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.372472 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db4a9b-0914-4c77-b645-f568f9460cf9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.418609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1143/2292] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:20.420725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db4a9b-0914-4c77-b645-f568f9460cf9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.430272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:20.492590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b2821-81cc-4ab7-852a-56a5d68f5765 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.525970 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db4a9b-0914-4c77-b645-f568f9460cf9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.531170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425ad68-f478-40e0-a503-c718ae650030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.531797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425ad68-f478-40e0-a503-c718ae650030 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.532075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425ad68-f478-40e0-a503-c718ae650030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.532418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425ad68-f478-40e0-a503-c718ae650030 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.533065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425ad68-f478-40e0-a503-c718ae650030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.535797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425ad68-f478-40e0-a503-c718ae650030 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.560891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1144/2293] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:20.562014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425ad68-f478-40e0-a503-c718ae650030 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.565344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d425ad68-f478-40e0-a503-c718ae650030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:20.566338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432168-560f-43c6-bc78-c8141f0d61e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.567391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432168-560f-43c6-bc78-c8141f0d61e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.567750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432168-560f-43c6-bc78-c8141f0d61e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.568097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432168-560f-43c6-bc78-c8141f0d61e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.568790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04432168-560f-43c6-bc78-c8141f0d61e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.570801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04432168-560f-43c6-bc78-c8141f0d61e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.579119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425ad68-f478-40e0-a503-c718ae650030 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.584937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.585561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.585917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.586195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] PATH_INFO: `/v3/projects/f5c82267f9224545a52eab89d60dd07e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.587582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c82267f9224545a52eab89d60dd07e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.589847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.600423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1150/2294] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/users => generated 299 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:20.601274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432168-560f-43c6-bc78-c8141f0d61e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.614479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432168-560f-43c6-bc78-c8141f0d61e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.619611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537f444-1213-4265-ab2c-6fae38b85f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.620256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537f444-1213-4265-ab2c-6fae38b85f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.620539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537f444-1213-4265-ab2c-6fae38b85f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.620923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537f444-1213-4265-ab2c-6fae38b85f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.621692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537f444-1213-4265-ab2c-6fae38b85f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.623401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537f444-1213-4265-ab2c-6fae38b85f6f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.636263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1151/2295] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:20.636964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537f444-1213-4265-ab2c-6fae38b85f6f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.661813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c82267f9224545a52eab89d60dd07e', 'request_id': 'req-c68e0405-82dd-428e-acd0-8f8329b3207a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.665599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537f444-1213-4265-ab2c-6fae38b85f6f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.678233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c82267f9224545a52eab89d60dd07e', 'request_id': 'req-c68e0405-82dd-428e-acd0-8f8329b3207a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.696317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c82267f9224545a52eab89d60dd07e', 'request_id': 'req-c68e0405-82dd-428e-acd0-8f8329b3207a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.719759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] The token cache is being invalidate because project f5c82267f9224545a52eab89d60dd07e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:20.720135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.721845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:20.724068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1145/2296] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:20 2022] DELETE /identity/v3/projects/f5c82267f9224545a52eab89d60dd07e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:20.724925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.726348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:20.758937 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d90238-474e-4213-9c98-8f04af65a39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.759616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d90238-474e-4213-9c98-8f04af65a39f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.759973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d90238-474e-4213-9c98-8f04af65a39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.760323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d90238-474e-4213-9c98-8f04af65a39f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.762032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d90238-474e-4213-9c98-8f04af65a39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.764525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d90238-474e-4213-9c98-8f04af65a39f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.781234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1152/2297] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:20.783501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facee83a-8795-4186-b24d-c4d9bf586940 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.784510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facee83a-8795-4186-b24d-c4d9bf586940 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.784855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facee83a-8795-4186-b24d-c4d9bf586940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.807899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-facee83a-8795-4186-b24d-c4d9bf586940 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:20.809282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-facee83a-8795-4186-b24d-c4d9bf586940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:20.816365 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e0405-82dd-428e-acd0-8f8329b3207a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.820352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1153/2298] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:20.821289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facee83a-8795-4186-b24d-c4d9bf586940 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.822776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-facee83a-8795-4186-b24d-c4d9bf586940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:20.827241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a5406-c64e-47ad-92a6-2f94d84765c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.827891 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a5406-c64e-47ad-92a6-2f94d84765c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.828171 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a5406-c64e-47ad-92a6-2f94d84765c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.828446 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a5406-c64e-47ad-92a6-2f94d84765c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.829428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a5406-c64e-47ad-92a6-2f94d84765c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.831447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a5406-c64e-47ad-92a6-2f94d84765c2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.837030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facee83a-8795-4186-b24d-c4d9bf586940 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.841861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ff16c-0034-4801-b2b7-c74a852c288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.842784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ff16c-0034-4801-b2b7-c74a852c288b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.845332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ff16c-0034-4801-b2b7-c74a852c288b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.845694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ff16c-0034-4801-b2b7-c74a852c288b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.847147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ff16c-0034-4801-b2b7-c74a852c288b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.850750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ff16c-0034-4801-b2b7-c74a852c288b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.855194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1146/2299] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:20.855194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06243d21-1a4c-4671-b5a1-74d0b3c0d065 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.855194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06243d21-1a4c-4671-b5a1-74d0b3c0d065 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.855194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06243d21-1a4c-4671-b5a1-74d0b3c0d065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.864334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1154/2300] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:20.865175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ff16c-0034-4801-b2b7-c74a852c288b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.871817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:20.876767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-06243d21-1a4c-4671-b5a1-74d0b3c0d065 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:20.878777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ff16c-0034-4801-b2b7-c74a852c288b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.878911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-06243d21-1a4c-4671-b5a1-74d0b3c0d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:20.886272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f648c30-5838-4be8-9e6b-f22b8432371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.887120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f648c30-5838-4be8-9e6b-f22b8432371b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.887630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f648c30-5838-4be8-9e6b-f22b8432371b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.887981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f648c30-5838-4be8-9e6b-f22b8432371b admin admin] PATH_INFO: `/v3/projects/a6a02f6df4ed4365b3ce1fa970f9cf41/users/334f2d8d9d9a4f7a90586999398f8a40/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.889525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1147/2301] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:20.890582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06243d21-1a4c-4671-b5a1-74d0b3c0d065 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.902164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06243d21-1a4c-4671-b5a1-74d0b3c0d065 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:20.909652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee766c-a008-47ac-b9ff-a5d75030fe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:20.910340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee766c-a008-47ac-b9ff-a5d75030fe7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.910698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee766c-a008-47ac-b9ff-a5d75030fe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.911161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee766c-a008-47ac-b9ff-a5d75030fe7d admin admin] PATH_INFO: `/v3/projects/ec4f7a3d6159462e969e32a5324c0d0f/users/b220a8309ded4e12bbb053848a4dc9df/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.913826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:20.919679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f648c30-5838-4be8-9e6b-f22b8432371b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a02f6df4ed4365b3ce1fa970f9cf41, user_id=334f2d8d9d9a4f7a90586999398f8a40, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.923280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f648c30-5838-4be8-9e6b-f22b8432371b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.930215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee766c-a008-47ac-b9ff-a5d75030fe7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4f7a3d6159462e969e32a5324c0d0f, user_id=b220a8309ded4e12bbb053848a4dc9df, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:20.933652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee766c-a008-47ac-b9ff-a5d75030fe7d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:20.938537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1155/2302] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:20 2022] PUT /identity/v3/projects/a6a02f6df4ed4365b3ce1fa970f9cf41/users/334f2d8d9d9a4f7a90586999398f8a40/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:20.939651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f648c30-5838-4be8-9e6b-f22b8432371b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:20.948262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1148/2303] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:20 2022] PUT /identity/v3/projects/ec4f7a3d6159462e969e32a5324c0d0f/users/b220a8309ded4e12bbb053848a4dc9df/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:20.954688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8a51-21ef-4670-972c-119ec33bde21 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:20.954688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8a51-21ef-4670-972c-119ec33bde21 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:20.954688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8a51-21ef-4670-972c-119ec33bde21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:20.974024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-192a8a51-21ef-4670-972c-119ec33bde21 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:20.975298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-192a8a51-21ef-4670-972c-119ec33bde21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:21.015527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f648c30-5838-4be8-9e6b-f22b8432371b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.052062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1149/2304] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:21.053420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e2888-870d-4d2a-8f11-0f44bfd206b1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.053705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e2888-870d-4d2a-8f11-0f44bfd206b1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.053974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e2888-870d-4d2a-8f11-0f44bfd206b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.081435 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-824e2888-870d-4d2a-8f11-0f44bfd206b1 None None] MFA Rules not processed for user `b220a8309ded4e12bbb053848a4dc9df`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:21.082740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-824e2888-870d-4d2a-8f11-0f44bfd206b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:21.084389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.085019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.085303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.085579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.086279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:21.086621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.088230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:21.100587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1156/2305] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:21.101159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:21.102869 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:21.156509 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1150/2306] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:21.157403 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e2888-870d-4d2a-8f11-0f44bfd206b1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:21.165998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbbb23-c85b-493a-841f-69e82c12f880 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.170028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e2888-870d-4d2a-8f11-0f44bfd206b1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.170485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56b8d8-5c8f-49f5-ab53-d4d011971251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.171134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56b8d8-5c8f-49f5-ab53-d4d011971251 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.171414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56b8d8-5c8f-49f5-ab53-d4d011971251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.171686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56b8d8-5c8f-49f5-ab53-d4d011971251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.175592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50a8c4-7d1e-4528-9535-be23f12148f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.176270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50a8c4-7d1e-4528-9535-be23f12148f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.176552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50a8c4-7d1e-4528-9535-be23f12148f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.176825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50a8c4-7d1e-4528-9535-be23f12148f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.177490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50a8c4-7d1e-4528-9535-be23f12148f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.179440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50a8c4-7d1e-4528-9535-be23f12148f5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:21.182601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56b8d8-5c8f-49f5-ab53-d4d011971251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.184038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56b8d8-5c8f-49f5-ab53-d4d011971251 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:21.193821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1157/2307] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:21.194725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56b8d8-5c8f-49f5-ab53-d4d011971251 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:21.196021 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb56b8d8-5c8f-49f5-ab53-d4d011971251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:21.213526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1151/2308] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:21 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:21.217563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50a8c4-7d1e-4528-9535-be23f12148f5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:21.228298 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:21.262212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56b8d8-5c8f-49f5-ab53-d4d011971251 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.298802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:21.314393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50a8c4-7d1e-4528-9535-be23f12148f5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.322178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287ef3e-4f7d-4f20-bb63-74e1a05bb1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.323056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287ef3e-4f7d-4f20-bb63-74e1a05bb1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.323502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287ef3e-4f7d-4f20-bb63-74e1a05bb1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.323893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287ef3e-4f7d-4f20-bb63-74e1a05bb1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.324710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287ef3e-4f7d-4f20-bb63-74e1a05bb1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.326889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287ef3e-4f7d-4f20-bb63-74e1a05bb1d0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:21.334024 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec002206-940b-4111-aa2a-ffea67526c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.334839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec002206-940b-4111-aa2a-ffea67526c7c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.335122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec002206-940b-4111-aa2a-ffea67526c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.335818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec002206-940b-4111-aa2a-ffea67526c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.351783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1152/2309] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:21 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:21.357583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287ef3e-4f7d-4f20-bb63-74e1a05bb1d0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:21.378863 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287ef3e-4f7d-4f20-bb63-74e1a05bb1d0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.384669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa925ec-eed5-45c8-86d8-180de92eeb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.385294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa925ec-eed5-45c8-86d8-180de92eeb3d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.385570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa925ec-eed5-45c8-86d8-180de92eeb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.385898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa925ec-eed5-45c8-86d8-180de92eeb3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.386567 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa925ec-eed5-45c8-86d8-180de92eeb3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.388162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa925ec-eed5-45c8-86d8-180de92eeb3d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:21.401032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1153/2310] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:21.404939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa925ec-eed5-45c8-86d8-180de92eeb3d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:21.414211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec002206-940b-4111-aa2a-ffea67526c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.415853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec002206-940b-4111-aa2a-ffea67526c7c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:21.417160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa925ec-eed5-45c8-86d8-180de92eeb3d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.423075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505f1fd-73c7-4228-a2e1-d3c298238c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.423763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505f1fd-73c7-4228-a2e1-d3c298238c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.424132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505f1fd-73c7-4228-a2e1-d3c298238c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.424543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505f1fd-73c7-4228-a2e1-d3c298238c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.425454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505f1fd-73c7-4228-a2e1-d3c298238c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.426506 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1158/2311] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:21.429015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505f1fd-73c7-4228-a2e1-d3c298238c98 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:21.452197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1154/2312] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:21.457164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec002206-940b-4111-aa2a-ffea67526c7c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:21.473709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec002206-940b-4111-aa2a-ffea67526c7c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.479162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf13ec-57a2-4e0f-bf14-93b0df4baba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.479921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf13ec-57a2-4e0f-bf14-93b0df4baba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.480247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf13ec-57a2-4e0f-bf14-93b0df4baba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.480652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf13ec-57a2-4e0f-bf14-93b0df4baba3 admin admin] PATH_INFO: `/v3/projects/8d9101e0f426415d9a56495feb50aec0/users/b3a7dc9c0ced4e87861b5450ebbc00b8/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.501513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf13ec-57a2-4e0f-bf14-93b0df4baba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9101e0f426415d9a56495feb50aec0, user_id=b3a7dc9c0ced4e87861b5450ebbc00b8, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.506727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf13ec-57a2-4e0f-bf14-93b0df4baba3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:21.534009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1159/2313] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:21 2022] PUT /identity/v3/projects/8d9101e0f426415d9a56495feb50aec0/users/b3a7dc9c0ced4e87861b5450ebbc00b8/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:00:21.541121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586ca29-1fef-4fbc-b8a2-2793839d5d1a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.541585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586ca29-1fef-4fbc-b8a2-2793839d5d1a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.542018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586ca29-1fef-4fbc-b8a2-2793839d5d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.598252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4586ca29-1fef-4fbc-b8a2-2793839d5d1a None None] MFA Rules not processed for user `b3a7dc9c0ced4e87861b5450ebbc00b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:21.600046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4586ca29-1fef-4fbc-b8a2-2793839d5d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:21.689599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1155/2314] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:21.694777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf13ec-57a2-4e0f-bf14-93b0df4baba3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:21.780387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf13ec-57a2-4e0f-bf14-93b0df4baba3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.791319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5021f0-ddea-4ebb-9078-d08b26fb5526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.792405 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5021f0-ddea-4ebb-9078-d08b26fb5526 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.792901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5021f0-ddea-4ebb-9078-d08b26fb5526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.793419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5021f0-ddea-4ebb-9078-d08b26fb5526 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.794309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5021f0-ddea-4ebb-9078-d08b26fb5526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.797143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5021f0-ddea-4ebb-9078-d08b26fb5526 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:21.815094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:21.838485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1160/2315] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:21 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:21.842354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586ca29-1fef-4fbc-b8a2-2793839d5d1a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:21.854884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586ca29-1fef-4fbc-b8a2-2793839d5d1a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:21.978340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058fbb4-5ea1-4ee0-ad26-2e24e4b9c79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:21.979330 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058fbb4-5ea1-4ee0-ad26-2e24e4b9c79a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:21.980087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058fbb4-5ea1-4ee0-ad26-2e24e4b9c79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:21.980338 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058fbb4-5ea1-4ee0-ad26-2e24e4b9c79a admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:21.981179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058fbb4-5ea1-4ee0-ad26-2e24e4b9c79a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:21.983907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058fbb4-5ea1-4ee0-ad26-2e24e4b9c79a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:22.020645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1156/2316] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:21 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:22.026726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5021f0-ddea-4ebb-9078-d08b26fb5526 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:22.041622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5021f0-ddea-4ebb-9078-d08b26fb5526 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:22.054974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb67d47-dc18-4b15-bf70-3d5d57802a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:22.056465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb67d47-dc18-4b15-bf70-3d5d57802a07 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:22.057395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb67d47-dc18-4b15-bf70-3d5d57802a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:22.058759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb67d47-dc18-4b15-bf70-3d5d57802a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:22.066602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb67d47-dc18-4b15-bf70-3d5d57802a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:22.068447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb67d47-dc18-4b15-bf70-3d5d57802a07 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:22.082256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1161/2317] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:22.086853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058fbb4-5ea1-4ee0-ad26-2e24e4b9c79a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:22.093696 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc9163-7b68-475d-a0d2-034ac5df78df None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:22.095549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc9163-7b68-475d-a0d2-034ac5df78df None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:22.097910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc9163-7b68-475d-a0d2-034ac5df78df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:22.102306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058fbb4-5ea1-4ee0-ad26-2e24e4b9c79a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:22.116913 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbef81b-8719-4515-b7ca-96655f0579e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:22.117817 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbef81b-8719-4515-b7ca-96655f0579e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:22.118202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbef81b-8719-4515-b7ca-96655f0579e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:22.119635 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbef81b-8719-4515-b7ca-96655f0579e5 admin admin] PATH_INFO: `/v3/projects/ba4be34881ef4e6ea5b80f10eef128c1/users/76efdd24759c48968b92f3522a14d0cf/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:22.151639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbef81b-8719-4515-b7ca-96655f0579e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4be34881ef4e6ea5b80f10eef128c1, user_id=76efdd24759c48968b92f3522a14d0cf, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:22.163161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5ddc9163-7b68-475d-a0d2-034ac5df78df None None] MFA Rules not processed for user `334f2d8d9d9a4f7a90586999398f8a40`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:22.165411 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5ddc9163-7b68-475d-a0d2-034ac5df78df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:22.167376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbef81b-8719-4515-b7ca-96655f0579e5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:22.189152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1157/2318] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:22 2022] PUT /identity/v3/projects/ba4be34881ef4e6ea5b80f10eef128c1/users/76efdd24759c48968b92f3522a14d0cf/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:22.198480 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbef81b-8719-4515-b7ca-96655f0579e5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:22.224023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:22.263125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1162/2319] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:22.276613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc9163-7b68-475d-a0d2-034ac5df78df None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:22.278082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5ddc9163-7b68-475d-a0d2-034ac5df78df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:22.296436 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbef81b-8719-4515-b7ca-96655f0579e5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:22.304536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a0ef7-6a13-457b-8221-3b88ed1ffd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:22.305298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a0ef7-6a13-457b-8221-3b88ed1ffd37 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:22.305627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a0ef7-6a13-457b-8221-3b88ed1ffd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:22.306021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a0ef7-6a13-457b-8221-3b88ed1ffd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:22.306822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a0ef7-6a13-457b-8221-3b88ed1ffd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:22.308561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a0ef7-6a13-457b-8221-3b88ed1ffd37 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:22.320783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1158/2320] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:22.324877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a0ef7-6a13-457b-8221-3b88ed1ffd37 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:22.337410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a0ef7-6a13-457b-8221-3b88ed1ffd37 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:22.343319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe67737-8282-409d-9259-893040f3ca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:22.343995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe67737-8282-409d-9259-893040f3ca80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:22.344307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe67737-8282-409d-9259-893040f3ca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:22.344583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe67737-8282-409d-9259-893040f3ca80 admin admin] PATH_INFO: `/v3/projects/ba4be34881ef4e6ea5b80f10eef128c1/users/76efdd24759c48968b92f3522a14d0cf/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:22.347310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe67737-8282-409d-9259-893040f3ca80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4be34881ef4e6ea5b80f10eef128c1, user_id=76efdd24759c48968b92f3522a14d0cf, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:22.350165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe67737-8282-409d-9259-893040f3ca80 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:22.361806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1159/2321] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:22 2022] PUT /identity/v3/projects/ba4be34881ef4e6ea5b80f10eef128c1/users/76efdd24759c48968b92f3522a14d0cf/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:22.372676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6f041-7e60-4832-bf31-5b9d63777044 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:22.372969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6f041-7e60-4832-bf31-5b9d63777044 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:22.373249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6f041-7e60-4832-bf31-5b9d63777044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:22.373384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc9163-7b68-475d-a0d2-034ac5df78df None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:22.403148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b3e6f041-7e60-4832-bf31-5b9d63777044 None None] MFA Rules not processed for user `76efdd24759c48968b92f3522a14d0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:22.404439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b3e6f041-7e60-4832-bf31-5b9d63777044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:22.430981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2a56a-d8b0-4216-9e3f-97f60f7a517b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:22.431607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2a56a-d8b0-4216-9e3f-97f60f7a517b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:22.431912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2a56a-d8b0-4216-9e3f-97f60f7a517b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:22.432356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2a56a-d8b0-4216-9e3f-97f60f7a517b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:22.497643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:22.510863 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1160/2322] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:22.520924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2a56a-d8b0-4216-9e3f-97f60f7a517b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:22.522382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2a56a-d8b0-4216-9e3f-97f60f7a517b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:22.531603 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1163/2323] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:22.716689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6f041-7e60-4832-bf31-5b9d63777044 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:22.739586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6f041-7e60-4832-bf31-5b9d63777044 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:22.750533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb87db-dd19-4a00-867d-ca20054d1474 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:22.751409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb87db-dd19-4a00-867d-ca20054d1474 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:22.751755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb87db-dd19-4a00-867d-ca20054d1474 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:22.752152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb87db-dd19-4a00-867d-ca20054d1474 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:22.789836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb87db-dd19-4a00-867d-ca20054d1474 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:22.791878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb87db-dd19-4a00-867d-ca20054d1474 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:22.808748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1161/2324] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:23.598960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2a56a-d8b0-4216-9e3f-97f60f7a517b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:23.662968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2a56a-d8b0-4216-9e3f-97f60f7a517b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:23.668398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:23.669069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:23.669352 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:23.669628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] PATH_INFO: `/v3/users/b3a7dc9c0ced4e87861b5450ebbc00b8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:23.671008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a7dc9c0ced4e87861b5450ebbc00b8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:23.672954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:23.719773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a7dc9c0ced4e87861b5450ebbc00b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:23.729175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a7dc9c0ced4e87861b5450ebbc00b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:23.729834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a7dc9c0ced4e87861b5450ebbc00b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:23.730448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a7dc9c0ced4e87861b5450ebbc00b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:23.743446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a7dc9c0ced4e87861b5450ebbc00b8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:23.757661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1164/2325] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:23 2022] DELETE /identity/v3/users/b3a7dc9c0ced4e87861b5450ebbc00b8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:23.762254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cf726-cf78-4465-aed9-e29c1cd2bed0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:23.762875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cf726-cf78-4465-aed9-e29c1cd2bed0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:23.762875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cf726-cf78-4465-aed9-e29c1cd2bed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:23.782184 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-725cf726-cf78-4465-aed9-e29c1cd2bed0 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:23.783533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-725cf726-cf78-4465-aed9-e29c1cd2bed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:23.850226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1162/2326] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:23.860510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:23.862070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:23.924833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1150b5-20a9-476c-8cef-4d9dcfe556dd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:23.928546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b8c8d-6492-4563-8d5f-f81a93723eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:23.929169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b8c8d-6492-4563-8d5f-f81a93723eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:23.929488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b8c8d-6492-4563-8d5f-f81a93723eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:23.929721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b8c8d-6492-4563-8d5f-f81a93723eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:23.941844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b8c8d-6492-4563-8d5f-f81a93723eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:23.943325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b8c8d-6492-4563-8d5f-f81a93723eb9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:23.962725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1165/2327] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:24.063662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5df41-74d6-4d2f-af56-8817e16da84e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:24.064020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5df41-74d6-4d2f-af56-8817e16da84e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:24.064377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5df41-74d6-4d2f-af56-8817e16da84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:24.089495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cec5df41-74d6-4d2f-af56-8817e16da84e None None] MFA Rules not processed for user `334f2d8d9d9a4f7a90586999398f8a40`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:24.091117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cec5df41-74d6-4d2f-af56-8817e16da84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:24.095257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:24.171008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1163/2328] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:24.310715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b8c8d-6492-4563-8d5f-f81a93723eb9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:24.312150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c65b8c8d-6492-4563-8d5f-f81a93723eb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:24.323450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:24.328170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b8c8d-6492-4563-8d5f-f81a93723eb9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:24.336184 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:24.337523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:24.338641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:24.339170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] PATH_INFO: `/v3/projects/8d9101e0f426415d9a56495feb50aec0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:24.340872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9101e0f426415d9a56495feb50aec0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:24.343809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:24.398110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9101e0f426415d9a56495feb50aec0', 'request_id': 'req-f5d31369-83f6-4ed6-add1-04e8cfda251c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.406325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9101e0f426415d9a56495feb50aec0', 'request_id': 'req-f5d31369-83f6-4ed6-add1-04e8cfda251c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.414076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9101e0f426415d9a56495feb50aec0', 'request_id': 'req-f5d31369-83f6-4ed6-add1-04e8cfda251c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.427827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] The token cache is being invalidate because project 8d9101e0f426415d9a56495feb50aec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:24.428143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.428815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.430485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1166/2329] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:24 2022] DELETE /identity/v3/projects/8d9101e0f426415d9a56495feb50aec0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:24.434273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5df41-74d6-4d2f-af56-8817e16da84e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:24.445471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5df41-74d6-4d2f-af56-8817e16da84e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:24.503483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:24.504126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:24.504407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:24.504681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] PATH_INFO: `/v3/users/76efdd24759c48968b92f3522a14d0cf` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:24.505849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=76efdd24759c48968b92f3522a14d0cf)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:24.507927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:24.561468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76efdd24759c48968b92f3522a14d0cf'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.568401 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76efdd24759c48968b92f3522a14d0cf'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.569069 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76efdd24759c48968b92f3522a14d0cf'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.569664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76efdd24759c48968b92f3522a14d0cf'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.582097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-52e4591f-7c5b-4a63-b0a3-68014e2312cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76efdd24759c48968b92f3522a14d0cf'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:24.595186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1164/2330] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:24 2022] DELETE /identity/v3/users/76efdd24759c48968b92f3522a14d0cf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:24.942242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:24.944224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:24.991398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d31369-83f6-4ed6-add1-04e8cfda251c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:25.082861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:25.083530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.083816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.084095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] PATH_INFO: `/v3/projects/ba4be34881ef4e6ea5b80f10eef128c1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.085282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba4be34881ef4e6ea5b80f10eef128c1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:25.087878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:25.156326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba4be34881ef4e6ea5b80f10eef128c1', 'request_id': 'req-da73a984-65a3-42de-b328-f062171909cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:25.165897 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba4be34881ef4e6ea5b80f10eef128c1', 'request_id': 'req-da73a984-65a3-42de-b328-f062171909cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:25.173190 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba4be34881ef4e6ea5b80f10eef128c1', 'request_id': 'req-da73a984-65a3-42de-b328-f062171909cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:25.185707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] The token cache is being invalidate because project ba4be34881ef4e6ea5b80f10eef128c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:25.186029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:25.186725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da73a984-65a3-42de-b328-f062171909cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:25.188384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1167/2331] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:24 2022] DELETE /identity/v3/projects/ba4be34881ef4e6ea5b80f10eef128c1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:25.197024 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dae6b-f780-4f15-ae93-7935eda0be5c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.197527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dae6b-f780-4f15-ae93-7935eda0be5c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.197637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dae6b-f780-4f15-ae93-7935eda0be5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.219444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7c5dae6b-f780-4f15-ae93-7935eda0be5c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:25.220690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7c5dae6b-f780-4f15-ae93-7935eda0be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:25.256361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:25.318769 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1165/2332] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:25.321322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312b41a-4dab-4d52-8646-c99f88bc3fb2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.321673 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312b41a-4dab-4d52-8646-c99f88bc3fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.322017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312b41a-4dab-4d52-8646-c99f88bc3fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.344748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a312b41a-4dab-4d52-8646-c99f88bc3fb2 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:25.346004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a312b41a-4dab-4d52-8646-c99f88bc3fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:25.355689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1168/2333] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:25.359418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dae6b-f780-4f15-ae93-7935eda0be5c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:25.370149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dae6b-f780-4f15-ae93-7935eda0be5c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:25.375348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a1171-9d74-4cde-99a3-e6549e4e64f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:25.375976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a1171-9d74-4cde-99a3-e6549e4e64f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.376255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a1171-9d74-4cde-99a3-e6549e4e64f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.376560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a1171-9d74-4cde-99a3-e6549e4e64f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.377265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a1171-9d74-4cde-99a3-e6549e4e64f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:25.378137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a1171-9d74-4cde-99a3-e6549e4e64f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:25.379805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a1171-9d74-4cde-99a3-e6549e4e64f9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:25.394224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1166/2334] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:25.397008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312b41a-4dab-4d52-8646-c99f88bc3fb2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:25.407067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312b41a-4dab-4d52-8646-c99f88bc3fb2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:25.414869 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6434a-dd00-4dad-806a-59f4e1ca1ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:25.415724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6434a-dd00-4dad-806a-59f4e1ca1ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.416073 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6434a-dd00-4dad-806a-59f4e1ca1ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.416433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6434a-dd00-4dad-806a-59f4e1ca1ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.417232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6434a-dd00-4dad-806a-59f4e1ca1ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:25.419622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6434a-dd00-4dad-806a-59f4e1ca1ca5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:25.435189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:25.456604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1169/2335] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:25 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:25.460829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a1171-9d74-4cde-99a3-e6549e4e64f9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:25.471605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a1171-9d74-4cde-99a3-e6549e4e64f9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:25.541862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca0e8d-c99c-4803-b6a2-75f1b8cd17ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:25.542412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca0e8d-c99c-4803-b6a2-75f1b8cd17ae admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.542720 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca0e8d-c99c-4803-b6a2-75f1b8cd17ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.542994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca0e8d-c99c-4803-b6a2-75f1b8cd17ae admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.543926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca0e8d-c99c-4803-b6a2-75f1b8cd17ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:25.545852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca0e8d-c99c-4803-b6a2-75f1b8cd17ae admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:25.568738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1167/2336] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:25 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:25.572972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6434a-dd00-4dad-806a-59f4e1ca1ca5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:25.584201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6434a-dd00-4dad-806a-59f4e1ca1ca5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:25.589005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff820ef-35c7-4d17-8a04-09475073844f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:25.589739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff820ef-35c7-4d17-8a04-09475073844f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.590079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff820ef-35c7-4d17-8a04-09475073844f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.590505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff820ef-35c7-4d17-8a04-09475073844f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.591558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff820ef-35c7-4d17-8a04-09475073844f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:25.593306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff820ef-35c7-4d17-8a04-09475073844f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:25.609032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1170/2337] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:25.613420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca0e8d-c99c-4803-b6a2-75f1b8cd17ae admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:25.623809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca0e8d-c99c-4803-b6a2-75f1b8cd17ae admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:25.630056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4d57c-66b7-4829-b5f9-ac711c1193bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:25.630769 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4d57c-66b7-4829-b5f9-ac711c1193bb admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.631003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4d57c-66b7-4829-b5f9-ac711c1193bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.631277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4d57c-66b7-4829-b5f9-ac711c1193bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.631974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4d57c-66b7-4829-b5f9-ac711c1193bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:25.633568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4d57c-66b7-4829-b5f9-ac711c1193bb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:25.646140 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1168/2338] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:25.650300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff820ef-35c7-4d17-8a04-09475073844f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:25.660789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff820ef-35c7-4d17-8a04-09475073844f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:25.666185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a26f7-b032-4f35-b82e-04efb29517b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:25.667034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a26f7-b032-4f35-b82e-04efb29517b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.667179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a26f7-b032-4f35-b82e-04efb29517b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.667557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a26f7-b032-4f35-b82e-04efb29517b7 admin admin] PATH_INFO: `/v3/projects/0550c47bc2d0445891fc6e260796d303/users/09bfff2f4c254858a00054dcb98f4743/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.685140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a26f7-b032-4f35-b82e-04efb29517b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0550c47bc2d0445891fc6e260796d303, user_id=09bfff2f4c254858a00054dcb98f4743, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:25.688022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a26f7-b032-4f35-b82e-04efb29517b7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:25.702022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1171/2339] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:25 2022] PUT /identity/v3/projects/0550c47bc2d0445891fc6e260796d303/users/09bfff2f4c254858a00054dcb98f4743/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:25.714707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf0638-5976-404d-8bac-f9419c0d6fc7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.715075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf0638-5976-404d-8bac-f9419c0d6fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.715422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf0638-5976-404d-8bac-f9419c0d6fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.748142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-03bf0638-5976-404d-8bac-f9419c0d6fc7 None None] MFA Rules not processed for user `09bfff2f4c254858a00054dcb98f4743`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:25.749351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-03bf0638-5976-404d-8bac-f9419c0d6fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:25.824372 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1169/2340] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:25.828881 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a26f7-b032-4f35-b82e-04efb29517b7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:25.890978 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a26f7-b032-4f35-b82e-04efb29517b7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:25.896316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c55714-dd89-49fe-a0b5-3d2066e912b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:25.896931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c55714-dd89-49fe-a0b5-3d2066e912b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.897214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c55714-dd89-49fe-a0b5-3d2066e912b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:25.897490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c55714-dd89-49fe-a0b5-3d2066e912b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:25.898152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c55714-dd89-49fe-a0b5-3d2066e912b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:25.900097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c55714-dd89-49fe-a0b5-3d2066e912b8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:25.926703 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1172/2341] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:25 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:25.930721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf0638-5976-404d-8bac-f9419c0d6fc7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:25.942197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf0638-5976-404d-8bac-f9419c0d6fc7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:25.967903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:25.998973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747b4b6-9b1a-4c56-9f89-bc1a40edafd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:25.999591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747b4b6-9b1a-4c56-9f89-bc1a40edafd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:25.999872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747b4b6-9b1a-4c56-9f89-bc1a40edafd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:26.000147 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747b4b6-9b1a-4c56-9f89-bc1a40edafd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:26.000818 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747b4b6-9b1a-4c56-9f89-bc1a40edafd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:26.003515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747b4b6-9b1a-4c56-9f89-bc1a40edafd3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:26.027097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1170/2342] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:25 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:26.030985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c55714-dd89-49fe-a0b5-3d2066e912b8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:26.041739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c55714-dd89-49fe-a0b5-3d2066e912b8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:26.046657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c568c-8bd3-4334-b6c7-db0696fc74c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:26.047270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c568c-8bd3-4334-b6c7-db0696fc74c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:26.047549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c568c-8bd3-4334-b6c7-db0696fc74c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:26.047819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c568c-8bd3-4334-b6c7-db0696fc74c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:26.048510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c568c-8bd3-4334-b6c7-db0696fc74c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:26.050720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c568c-8bd3-4334-b6c7-db0696fc74c5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:26.061346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1173/2343] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:00:26.066130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747b4b6-9b1a-4c56-9f89-bc1a40edafd3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:26.076132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747b4b6-9b1a-4c56-9f89-bc1a40edafd3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:26.080837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a151c-32a0-4b0f-9267-d1cdf47fff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:26.081493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a151c-32a0-4b0f-9267-d1cdf47fff32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:26.081769 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a151c-32a0-4b0f-9267-d1cdf47fff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:26.082038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a151c-32a0-4b0f-9267-d1cdf47fff32 admin admin] PATH_INFO: `/v3/projects/df4096b2c909472388fea2b5c11ff376/users/16da1012ae6943c0b97d22ca51b027f5/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:26.096609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a151c-32a0-4b0f-9267-d1cdf47fff32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4096b2c909472388fea2b5c11ff376, user_id=16da1012ae6943c0b97d22ca51b027f5, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:26.099305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a151c-32a0-4b0f-9267-d1cdf47fff32 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:26.112563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1171/2344] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:26 2022] PUT /identity/v3/projects/df4096b2c909472388fea2b5c11ff376/users/16da1012ae6943c0b97d22ca51b027f5/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:26.116482 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c568c-8bd3-4334-b6c7-db0696fc74c5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:26.126297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c568c-8bd3-4334-b6c7-db0696fc74c5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:26.185613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4fe20-16c9-4e9c-98b5-9771122648a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:26.186236 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4fe20-16c9-4e9c-98b5-9771122648a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:26.186622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4fe20-16c9-4e9c-98b5-9771122648a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:26.186929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4fe20-16c9-4e9c-98b5-9771122648a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:26.187654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4fe20-16c9-4e9c-98b5-9771122648a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:26.189309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4fe20-16c9-4e9c-98b5-9771122648a1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:26.203260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1174/2345] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:26.204680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a151c-32a0-4b0f-9267-d1cdf47fff32 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:26.221700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a151c-32a0-4b0f-9267-d1cdf47fff32 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:26.227001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ca617-c30f-44c6-8200-c4a177f9c12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:26.227741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ca617-c30f-44c6-8200-c4a177f9c12d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:26.228110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ca617-c30f-44c6-8200-c4a177f9c12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:26.228449 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ca617-c30f-44c6-8200-c4a177f9c12d admin admin] PATH_INFO: `/v3/projects/df4096b2c909472388fea2b5c11ff376/users/16da1012ae6943c0b97d22ca51b027f5/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:26.230504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ca617-c30f-44c6-8200-c4a177f9c12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4096b2c909472388fea2b5c11ff376, user_id=16da1012ae6943c0b97d22ca51b027f5, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:26.233459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ca617-c30f-44c6-8200-c4a177f9c12d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:26.244932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1172/2346] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:26 2022] PUT /identity/v3/projects/df4096b2c909472388fea2b5c11ff376/users/16da1012ae6943c0b97d22ca51b027f5/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:26.256087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74917f-83c6-499d-ba5d-23e8b1ac8353 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:26.256376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74917f-83c6-499d-ba5d-23e8b1ac8353 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:26.256653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74917f-83c6-499d-ba5d-23e8b1ac8353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:26.284376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dd74917f-83c6-499d-ba5d-23e8b1ac8353 None None] MFA Rules not processed for user `16da1012ae6943c0b97d22ca51b027f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:26.285592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd74917f-83c6-499d-ba5d-23e8b1ac8353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:26.298995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:26.360378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1175/2347] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:26.371791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ca617-c30f-44c6-8200-c4a177f9c12d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:26.373480 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-368ca617-c30f-44c6-8200-c4a177f9c12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:26.446117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ca617-c30f-44c6-8200-c4a177f9c12d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:26.449748 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd9586-210e-414f-a671-a60737735373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:26.450368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd9586-210e-414f-a671-a60737735373 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:26.450673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd9586-210e-414f-a671-a60737735373 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:26.450940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd9586-210e-414f-a671-a60737735373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:26.510334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd9586-210e-414f-a671-a60737735373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:26.511825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd9586-210e-414f-a671-a60737735373 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:26.521534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1173/2348] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:29.233483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74917f-83c6-499d-ba5d-23e8b1ac8353 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:29.306424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74917f-83c6-499d-ba5d-23e8b1ac8353 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:29.311063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:29.311730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:29.312009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:29.312282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] PATH_INFO: `/v3/users/09bfff2f4c254858a00054dcb98f4743` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:29.313483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09bfff2f4c254858a00054dcb98f4743)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:29.316063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:29.369445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09bfff2f4c254858a00054dcb98f4743'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.376484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09bfff2f4c254858a00054dcb98f4743'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.377227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09bfff2f4c254858a00054dcb98f4743'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.378452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09bfff2f4c254858a00054dcb98f4743'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.390941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09bfff2f4c254858a00054dcb98f4743'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.405910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1176/2349] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:29 2022] DELETE /identity/v3/users/09bfff2f4c254858a00054dcb98f4743 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:29.410465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e539871-c98c-4f83-a837-0dd5edccc98f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:29.410988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e539871-c98c-4f83-a837-0dd5edccc98f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:29.411104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e539871-c98c-4f83-a837-0dd5edccc98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:29.431230 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3e539871-c98c-4f83-a837-0dd5edccc98f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:29.432456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3e539871-c98c-4f83-a837-0dd5edccc98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:29.464221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:29.504601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1174/2350] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:29.514470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:29.516026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:29.553522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:29.579919 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bdab2-64e1-497b-94a3-b1f5211dd972 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:29.584164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dffe6-659f-494f-b5d8-6f2c7c9a784b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:29.584945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dffe6-659f-494f-b5d8-6f2c7c9a784b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:29.585312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dffe6-659f-494f-b5d8-6f2c7c9a784b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:29.585592 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dffe6-659f-494f-b5d8-6f2c7c9a784b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:29.595292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dffe6-659f-494f-b5d8-6f2c7c9a784b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:29.596738 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dffe6-659f-494f-b5d8-6f2c7c9a784b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:29.606871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1177/2351] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:29.783768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e539871-c98c-4f83-a837-0dd5edccc98f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:29.787137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3e539871-c98c-4f83-a837-0dd5edccc98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:29.803438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e539871-c98c-4f83-a837-0dd5edccc98f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:29.808653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:29.809538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:29.809993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:29.810339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] PATH_INFO: `/v3/users/b220a8309ded4e12bbb053848a4dc9df` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:29.812415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b220a8309ded4e12bbb053848a4dc9df)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:29.814841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:29.868906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b220a8309ded4e12bbb053848a4dc9df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.880328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b220a8309ded4e12bbb053848a4dc9df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.882156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b220a8309ded4e12bbb053848a4dc9df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.882424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b220a8309ded4e12bbb053848a4dc9df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.897858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b220a8309ded4e12bbb053848a4dc9df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:29.915259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1175/2352] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:29 2022] DELETE /identity/v3/users/b220a8309ded4e12bbb053848a4dc9df => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:29.917233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc912038-5481-4abe-b4d0-c9a3498b567c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:29.917628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc912038-5481-4abe-b4d0-c9a3498b567c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:29.917911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc912038-5481-4abe-b4d0-c9a3498b567c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:29.938916 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fc912038-5481-4abe-b4d0-c9a3498b567c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:29.940718 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc912038-5481-4abe-b4d0-c9a3498b567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:30.006983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:30.008414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:30.055532 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1178/2353] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:30.072838 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc912038-5481-4abe-b4d0-c9a3498b567c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:30.075543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc912038-5481-4abe-b4d0-c9a3498b567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:30.140814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728361d-7145-41e0-a900-baaf7ac25789 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:30.148902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:30.149709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:30.150103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:30.150559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] PATH_INFO: `/v3/projects/0550c47bc2d0445891fc6e260796d303` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:30.152302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0550c47bc2d0445891fc6e260796d303)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:30.156082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:30.196486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc912038-5481-4abe-b4d0-c9a3498b567c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:30.201482 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bbd3d-4abb-485b-8612-1217091620bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:30.202122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bbd3d-4abb-485b-8612-1217091620bf service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:30.202371 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bbd3d-4abb-485b-8612-1217091620bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:30.202657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bbd3d-4abb-485b-8612-1217091620bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:30.229241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0550c47bc2d0445891fc6e260796d303', 'request_id': 'req-a9000b12-1471-42d9-b795-e05070e0dfd6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.231208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:30.234849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bbd3d-4abb-485b-8612-1217091620bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:30.237247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bbd3d-4abb-485b-8612-1217091620bf service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:30.243753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0550c47bc2d0445891fc6e260796d303', 'request_id': 'req-a9000b12-1471-42d9-b795-e05070e0dfd6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.253790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0550c47bc2d0445891fc6e260796d303', 'request_id': 'req-a9000b12-1471-42d9-b795-e05070e0dfd6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.261316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1179/2354] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:30.269560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] The token cache is being invalidate because project 0550c47bc2d0445891fc6e260796d303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:30.270047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.270806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.272930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1176/2355] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:30 2022] DELETE /identity/v3/projects/0550c47bc2d0445891fc6e260796d303 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:30.276751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bbd3d-4abb-485b-8612-1217091620bf service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:30.289426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bbd3d-4abb-485b-8612-1217091620bf service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:30.356346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:30.357005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:30.357283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:30.357623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] PATH_INFO: `/v3/users/16da1012ae6943c0b97d22ca51b027f5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:30.358902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16da1012ae6943c0b97d22ca51b027f5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:30.360866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:30.379870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:30.422742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16da1012ae6943c0b97d22ca51b027f5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.444659 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16da1012ae6943c0b97d22ca51b027f5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.445665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16da1012ae6943c0b97d22ca51b027f5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.446301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16da1012ae6943c0b97d22ca51b027f5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.466579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16da1012ae6943c0b97d22ca51b027f5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:30.485579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1180/2356] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:30 2022] DELETE /identity/v3/users/16da1012ae6943c0b97d22ca51b027f5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:30.942686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:30.944274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:30.964820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9000b12-1471-42d9-b795-e05070e0dfd6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:31.057984 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:31.058762 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.059388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.059503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] PATH_INFO: `/v3/projects/df4096b2c909472388fea2b5c11ff376` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.060775 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=df4096b2c909472388fea2b5c11ff376)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:31.062902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:31.079384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:31.081608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:31.101796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb2e3e-0133-4932-8732-9d69b75ddd90 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:31.107749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:31.108416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.108679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.109110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] PATH_INFO: `/v3/projects/ec4f7a3d6159462e969e32a5324c0d0f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.110298 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4f7a3d6159462e969e32a5324c0d0f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:31.113587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:31.187315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df4096b2c909472388fea2b5c11ff376', 'request_id': 'req-47c93c15-7df5-4395-a24d-e2389da5b3cf'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.217038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df4096b2c909472388fea2b5c11ff376', 'request_id': 'req-47c93c15-7df5-4395-a24d-e2389da5b3cf'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.226858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df4096b2c909472388fea2b5c11ff376', 'request_id': 'req-47c93c15-7df5-4395-a24d-e2389da5b3cf'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.233286 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4f7a3d6159462e969e32a5324c0d0f', 'request_id': 'req-cee5219a-6ab2-438e-854d-eaef80ac409d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.241145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] The token cache is being invalidate because project df4096b2c909472388fea2b5c11ff376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:31.242364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.242573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4f7a3d6159462e969e32a5324c0d0f', 'request_id': 'req-cee5219a-6ab2-438e-854d-eaef80ac409d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.243322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-47c93c15-7df5-4395-a24d-e2389da5b3cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.246559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1177/2357] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:30 2022] DELETE /identity/v3/projects/df4096b2c909472388fea2b5c11ff376 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:31.248402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4f7a3d6159462e969e32a5324c0d0f', 'request_id': 'req-cee5219a-6ab2-438e-854d-eaef80ac409d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.250077 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc397f-b0e2-44ff-a5bc-631fd190133d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.250563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc397f-b0e2-44ff-a5bc-631fd190133d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.250903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc397f-b0e2-44ff-a5bc-631fd190133d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.259931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] The token cache is being invalidate because project ec4f7a3d6159462e969e32a5324c0d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:31.260290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.260954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cee5219a-6ab2-438e-854d-eaef80ac409d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:31.263515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1181/2358] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:31 2022] DELETE /identity/v3/projects/ec4f7a3d6159462e969e32a5324c0d0f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:31.274361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1afc397f-b0e2-44ff-a5bc-631fd190133d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:31.275696 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1afc397f-b0e2-44ff-a5bc-631fd190133d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:31.348800 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1178/2359] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:31.357716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d505-c68b-4fd6-a7a7-d8d21aab95ee None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.357716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d505-c68b-4fd6-a7a7-d8d21aab95ee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.358422 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d505-c68b-4fd6-a7a7-d8d21aab95ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.386238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dd73d505-c68b-4fd6-a7a7-d8d21aab95ee None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:31.391779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd73d505-c68b-4fd6-a7a7-d8d21aab95ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:31.406180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1182/2360] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:31.407578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc397f-b0e2-44ff-a5bc-631fd190133d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:31.420771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc397f-b0e2-44ff-a5bc-631fd190133d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:31.440547 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef59756-2c12-4669-a0d6-27314ccb02c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:31.441380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef59756-2c12-4669-a0d6-27314ccb02c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.441661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef59756-2c12-4669-a0d6-27314ccb02c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.441931 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef59756-2c12-4669-a0d6-27314ccb02c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.443351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef59756-2c12-4669-a0d6-27314ccb02c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:31.443658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef59756-2c12-4669-a0d6-27314ccb02c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:31.445359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef59756-2c12-4669-a0d6-27314ccb02c1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:31.448590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:31.467808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1179/2361] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:31.473192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d505-c68b-4fd6-a7a7-d8d21aab95ee None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:31.484791 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d505-c68b-4fd6-a7a7-d8d21aab95ee None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:31.491115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4130e77-d6df-4f8f-af23-89a64a5979f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:31.492340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4130e77-d6df-4f8f-af23-89a64a5979f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.492836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4130e77-d6df-4f8f-af23-89a64a5979f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.493343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4130e77-d6df-4f8f-af23-89a64a5979f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.494242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4130e77-d6df-4f8f-af23-89a64a5979f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:31.496562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4130e77-d6df-4f8f-af23-89a64a5979f5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:31.537787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1183/2362] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:31 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:31.543175 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef59756-2c12-4669-a0d6-27314ccb02c1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:31.556690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef59756-2c12-4669-a0d6-27314ccb02c1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:31.603356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4130e77-d6df-4f8f-af23-89a64a5979f5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:31.605447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4130e77-d6df-4f8f-af23-89a64a5979f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:31.647993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e357d-cf7b-411a-a322-fc911b4106c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:31.649213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e357d-cf7b-411a-a322-fc911b4106c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.649644 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e357d-cf7b-411a-a322-fc911b4106c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.650093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e357d-cf7b-411a-a322-fc911b4106c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.652686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e357d-cf7b-411a-a322-fc911b4106c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:31.654694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e357d-cf7b-411a-a322-fc911b4106c8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:31.690780 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1180/2363] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:31 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:31.694369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e357d-cf7b-411a-a322-fc911b4106c8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:31.709487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:31.712406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e357d-cf7b-411a-a322-fc911b4106c8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:31.719390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0da845-49ce-43dd-a5fe-512e48b0aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:31.720094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0da845-49ce-43dd-a5fe-512e48b0aae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.720707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0da845-49ce-43dd-a5fe-512e48b0aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.720987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0da845-49ce-43dd-a5fe-512e48b0aae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.722715 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0da845-49ce-43dd-a5fe-512e48b0aae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:31.725150 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0da845-49ce-43dd-a5fe-512e48b0aae5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:31.725953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4130e77-d6df-4f8f-af23-89a64a5979f5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:31.737721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee927981-33ae-44b1-b322-bdec986c02f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:31.738658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee927981-33ae-44b1-b322-bdec986c02f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.738941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee927981-33ae-44b1-b322-bdec986c02f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.739215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee927981-33ae-44b1-b322-bdec986c02f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.741023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee927981-33ae-44b1-b322-bdec986c02f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:31.750872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1181/2364] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:31.755728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0da845-49ce-43dd-a5fe-512e48b0aae5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:31.770827 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0da845-49ce-43dd-a5fe-512e48b0aae5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:31.776889 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028715fb-d78f-4fd7-b279-1a06525f8424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:31.778926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028715fb-d78f-4fd7-b279-1a06525f8424 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.779289 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028715fb-d78f-4fd7-b279-1a06525f8424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.779654 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028715fb-d78f-4fd7-b279-1a06525f8424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.780906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028715fb-d78f-4fd7-b279-1a06525f8424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:31.782842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028715fb-d78f-4fd7-b279-1a06525f8424 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:31.804098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1182/2365] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:31.808565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028715fb-d78f-4fd7-b279-1a06525f8424 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:31.822958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028715fb-d78f-4fd7-b279-1a06525f8424 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:31.828242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476120b1-a6a9-43a3-b6e8-d7a794150bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:31.829594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476120b1-a6a9-43a3-b6e8-d7a794150bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.829879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476120b1-a6a9-43a3-b6e8-d7a794150bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.830171 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476120b1-a6a9-43a3-b6e8-d7a794150bd6 admin admin] PATH_INFO: `/v3/projects/f102af4a74e946caa9fefc7ad0ae148c/users/b98e41589d15488380a43000a82c2b3c/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.839041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee927981-33ae-44b1-b322-bdec986c02f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:31.841149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee927981-33ae-44b1-b322-bdec986c02f6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:31.848859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476120b1-a6a9-43a3-b6e8-d7a794150bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f102af4a74e946caa9fefc7ad0ae148c, user_id=b98e41589d15488380a43000a82c2b3c, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:31.852306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1184/2366] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:31.852588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476120b1-a6a9-43a3-b6e8-d7a794150bd6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:31.874051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1183/2367] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:31 2022] PUT /identity/v3/projects/f102af4a74e946caa9fefc7ad0ae148c/users/b98e41589d15488380a43000a82c2b3c/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:31.880211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec26c8-1209-4f73-ade8-81d263bf2010 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:31.880505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec26c8-1209-4f73-ade8-81d263bf2010 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:31.880784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec26c8-1209-4f73-ade8-81d263bf2010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:31.902139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-beec26c8-1209-4f73-ade8-81d263bf2010 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:31.903258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-beec26c8-1209-4f73-ade8-81d263bf2010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:31.993299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1185/2368] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:32.011443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476120b1-a6a9-43a3-b6e8-d7a794150bd6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:32.099922 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476120b1-a6a9-43a3-b6e8-d7a794150bd6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:32.106338 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241be50-5c14-462d-9f59-984c97515cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:32.107257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241be50-5c14-462d-9f59-984c97515cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:32.107615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241be50-5c14-462d-9f59-984c97515cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:32.107902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241be50-5c14-462d-9f59-984c97515cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:32.119918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241be50-5c14-462d-9f59-984c97515cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:32.121361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241be50-5c14-462d-9f59-984c97515cc7 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:32.135758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1184/2369] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:33.648089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922ab2a-8c4c-4112-82ff-1b0c910aecef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:33.648552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922ab2a-8c4c-4112-82ff-1b0c910aecef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:33.648658 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922ab2a-8c4c-4112-82ff-1b0c910aecef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:33.689066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6922ab2a-8c4c-4112-82ff-1b0c910aecef None None] MFA Rules not processed for user `b98e41589d15488380a43000a82c2b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:33.691336 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6922ab2a-8c4c-4112-82ff-1b0c910aecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:33.790247 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1186/2370] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:33.808737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241be50-5c14-462d-9f59-984c97515cc7 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:33.810239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0241be50-5c14-462d-9f59-984c97515cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:33.910008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:33.924131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241be50-5c14-462d-9f59-984c97515cc7 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:33.936490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f39a56-21d6-4f20-8267-8ee3a8ec9881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:33.937301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39a56-21d6-4f20-8267-8ee3a8ec9881 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:33.937579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39a56-21d6-4f20-8267-8ee3a8ec9881 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:33.937852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f39a56-21d6-4f20-8267-8ee3a8ec9881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:33.947963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f39a56-21d6-4f20-8267-8ee3a8ec9881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:33.949544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f39a56-21d6-4f20-8267-8ee3a8ec9881 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:33.963074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1185/2371] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:34.919017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb0445-1a9d-450a-b729-e390fce5b1f0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:34.919587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb0445-1a9d-450a-b729-e390fce5b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:34.919587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb0445-1a9d-450a-b729-e390fce5b1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:34.941459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-94bb0445-1a9d-450a-b729-e390fce5b1f0 None None] MFA Rules not processed for user `a8abb58f9c274e568d2d0fe3d4df11de`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:34.942704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94bb0445-1a9d-450a-b729-e390fce5b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:34.970262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:35.006682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1187/2372] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:35.395764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdc7d9-0f41-438c-8dc9-f244d95fdd74 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:35.396637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdc7d9-0f41-438c-8dc9-f244d95fdd74 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:35.397110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdc7d9-0f41-438c-8dc9-f244d95fdd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:35.417151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-eecdc7d9-0f41-438c-8dc9-f244d95fdd74 None None] MFA Rules not processed for user `b98e41589d15488380a43000a82c2b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:35.418404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eecdc7d9-0f41-438c-8dc9-f244d95fdd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:35.428229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1186/2373] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:43.771429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb0445-1a9d-450a-b729-e390fce5b1f0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:43.773207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94bb0445-1a9d-450a-b729-e390fce5b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:43.789443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb0445-1a9d-450a-b729-e390fce5b1f0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:43.796143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:43.796844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:43.797126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:43.797536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] PATH_INFO: `/v3/users/5df6d206b9c14d3d916fe2d3449ed93a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:43.798782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df6d206b9c14d3d916fe2d3449ed93a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:43.800801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:43.880961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df6d206b9c14d3d916fe2d3449ed93a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:43.890314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df6d206b9c14d3d916fe2d3449ed93a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:43.891106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df6d206b9c14d3d916fe2d3449ed93a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:43.892134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df6d206b9c14d3d916fe2d3449ed93a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:43.918646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-09e717ae-7bc5-4f6a-b7a4-71602dbea702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df6d206b9c14d3d916fe2d3449ed93a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:43.934857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1188/2374] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:00:43 2022] DELETE /identity/v3/users/5df6d206b9c14d3d916fe2d3449ed93a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:43.939464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5937a22-0ce1-4204-9159-bf0e2de4addc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:43.939974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5937a22-0ce1-4204-9159-bf0e2de4addc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:43.940237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5937a22-0ce1-4204-9159-bf0e2de4addc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:43.961512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e5937a22-0ce1-4204-9159-bf0e2de4addc None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:43.962813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e5937a22-0ce1-4204-9159-bf0e2de4addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:44.070883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1187/2375] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:44.078676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561cd9b-1686-440f-9203-d566feff89aa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:44.078997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561cd9b-1686-440f-9203-d566feff89aa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:44.079301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561cd9b-1686-440f-9203-d566feff89aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:44.081224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5937a22-0ce1-4204-9159-bf0e2de4addc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:44.082659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e5937a22-0ce1-4204-9159-bf0e2de4addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:44.104323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2561cd9b-1686-440f-9203-d566feff89aa None None] MFA Rules not processed for user `25db6f7402ae4a14a04fab0a9915ab77`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:44.105529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2561cd9b-1686-440f-9203-d566feff89aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:44.142808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5937a22-0ce1-4204-9159-bf0e2de4addc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:44.147399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869d9ff-f753-4ae5-9d5b-f1d762dce3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:44.148015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869d9ff-f753-4ae5-9d5b-f1d762dce3ae service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:44.148291 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869d9ff-f753-4ae5-9d5b-f1d762dce3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:44.148559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869d9ff-f753-4ae5-9d5b-f1d762dce3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:44.158215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869d9ff-f753-4ae5-9d5b-f1d762dce3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:44.160328 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869d9ff-f753-4ae5-9d5b-f1d762dce3ae service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:44.167219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1189/2376] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:44.171465 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1188/2377] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:00:44.595327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561cd9b-1686-440f-9203-d566feff89aa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:44.596891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2561cd9b-1686-440f-9203-d566feff89aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:44.617036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2561cd9b-1686-440f-9203-d566feff89aa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:44.630859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:44.631528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:44.631809 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:44.632083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] PATH_INFO: `/v3/projects/847627489c2d483e8594b9bd848f6e95` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:44.633286 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=847627489c2d483e8594b9bd848f6e95)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:44.635514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:44.691079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:44.695401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '847627489c2d483e8594b9bd848f6e95', 'request_id': 'req-043f497f-7701-483b-8c9d-41fac6f1a5ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:44.705883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '847627489c2d483e8594b9bd848f6e95', 'request_id': 'req-043f497f-7701-483b-8c9d-41fac6f1a5ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:44.712470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '847627489c2d483e8594b9bd848f6e95', 'request_id': 'req-043f497f-7701-483b-8c9d-41fac6f1a5ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:44.725413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] The token cache is being invalidate because project 847627489c2d483e8594b9bd848f6e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:00:44.725708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:44.726408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-043f497f-7701-483b-8c9d-41fac6f1a5ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:00:44.727955 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1190/2378] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:00:44 2022] DELETE /identity/v3/projects/847627489c2d483e8594b9bd848f6e95 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:44.733015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced60441-c15d-4488-b55d-1ded3697a42c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:44.733305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced60441-c15d-4488-b55d-1ded3697a42c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:44.733583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced60441-c15d-4488-b55d-1ded3697a42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:44.754255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ced60441-c15d-4488-b55d-1ded3697a42c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:44.755512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ced60441-c15d-4488-b55d-1ded3697a42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:44.830158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1189/2379] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:44.838423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1384-ade3-4b37-84ef-88b453c96f49 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:44.838954 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1384-ade3-4b37-84ef-88b453c96f49 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:44.839049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e1384-ade3-4b37-84ef-88b453c96f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:44.866835 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-353e1384-ade3-4b37-84ef-88b453c96f49 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:44.868063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-353e1384-ade3-4b37-84ef-88b453c96f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:44.876887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1191/2380] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:00:44.883563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced60441-c15d-4488-b55d-1ded3697a42c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:44.895523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced60441-c15d-4488-b55d-1ded3697a42c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:44.911056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4725d43-db2f-469f-9793-6230a4265425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:44.911723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4725d43-db2f-469f-9793-6230a4265425 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:44.912006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4725d43-db2f-469f-9793-6230a4265425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:44.912283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4725d43-db2f-469f-9793-6230a4265425 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:44.912990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4725d43-db2f-469f-9793-6230a4265425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:00:44.913296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4725d43-db2f-469f-9793-6230a4265425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:44.914956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4725d43-db2f-469f-9793-6230a4265425 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:44.937381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1190/2381] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:00:44.942426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1384-ade3-4b37-84ef-88b453c96f49 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:44.969630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e1384-ade3-4b37-84ef-88b453c96f49 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:44.975051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5969f69-ad8e-4fe5-91ae-0079df0bc31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:44.975691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969f69-ad8e-4fe5-91ae-0079df0bc31e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:44.976053 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969f69-ad8e-4fe5-91ae-0079df0bc31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:44.976541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5969f69-ad8e-4fe5-91ae-0079df0bc31e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:44.977279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5969f69-ad8e-4fe5-91ae-0079df0bc31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:44.979250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5969f69-ad8e-4fe5-91ae-0079df0bc31e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:45.022161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1192/2382] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:44 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:45.022853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4725d43-db2f-469f-9793-6230a4265425 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:45.036857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4725d43-db2f-469f-9793-6230a4265425 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:45.043533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:45.120796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2405e-3a6c-444d-ad50-7e9b2408e291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:45.121846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2405e-3a6c-444d-ad50-7e9b2408e291 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:45.122258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2405e-3a6c-444d-ad50-7e9b2408e291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:45.122646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2405e-3a6c-444d-ad50-7e9b2408e291 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:45.123467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2405e-3a6c-444d-ad50-7e9b2408e291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:45.125703 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2405e-3a6c-444d-ad50-7e9b2408e291 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:45.168047 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1191/2383] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:45 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:00:45.172514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5969f69-ad8e-4fe5-91ae-0079df0bc31e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:45.184529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5969f69-ad8e-4fe5-91ae-0079df0bc31e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:45.189394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901ab56-cd88-49e4-8de5-9f8e99883292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:45.190583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901ab56-cd88-49e4-8de5-9f8e99883292 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:45.190872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901ab56-cd88-49e4-8de5-9f8e99883292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:45.191216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901ab56-cd88-49e4-8de5-9f8e99883292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:45.191863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901ab56-cd88-49e4-8de5-9f8e99883292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:45.193486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2901ab56-cd88-49e4-8de5-9f8e99883292 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:45.204083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1193/2384] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:45.209163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2405e-3a6c-444d-ad50-7e9b2408e291 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:45.223314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2405e-3a6c-444d-ad50-7e9b2408e291 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:45.232778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a693af61-7f1c-4c25-a54c-70eb99f8825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:45.233585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693af61-7f1c-4c25-a54c-70eb99f8825f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:45.234108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693af61-7f1c-4c25-a54c-70eb99f8825f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:45.234681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a693af61-7f1c-4c25-a54c-70eb99f8825f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:45.235653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693af61-7f1c-4c25-a54c-70eb99f8825f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:45.238281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a693af61-7f1c-4c25-a54c-70eb99f8825f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:45.252243 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1192/2385] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:45.255983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901ab56-cd88-49e4-8de5-9f8e99883292 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:45.270971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901ab56-cd88-49e4-8de5-9f8e99883292 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:45.276642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e9d02-f923-431f-9761-e6dfa0bffdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:45.277534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e9d02-f923-431f-9761-e6dfa0bffdda admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:45.277820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e9d02-f923-431f-9761-e6dfa0bffdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:45.278101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e9d02-f923-431f-9761-e6dfa0bffdda admin admin] PATH_INFO: `/v3/projects/db365a6a9f6041a7a8749fd1a1e3d488/users/407ad282edd74fb1adf62fb63306779c/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:45.297168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e9d02-f923-431f-9761-e6dfa0bffdda admin admin] RBAC: Authorizing `identity:create_grant(project_id=db365a6a9f6041a7a8749fd1a1e3d488, user_id=407ad282edd74fb1adf62fb63306779c, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:45.300777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e9d02-f923-431f-9761-e6dfa0bffdda admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:45.315194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1194/2386] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:45 2022] PUT /identity/v3/projects/db365a6a9f6041a7a8749fd1a1e3d488/users/407ad282edd74fb1adf62fb63306779c/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:45.320456 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176369d-a323-41fe-b9b0-7c769ccdbec1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:45.320751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176369d-a323-41fe-b9b0-7c769ccdbec1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:45.321073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176369d-a323-41fe-b9b0-7c769ccdbec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:45.343582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9176369d-a323-41fe-b9b0-7c769ccdbec1 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:45.345014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9176369d-a323-41fe-b9b0-7c769ccdbec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:45.408063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1193/2387] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:45.419680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e9d02-f923-431f-9761-e6dfa0bffdda admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:45.421468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f0e9d02-f923-431f-9761-e6dfa0bffdda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:45.482113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e9d02-f923-431f-9761-e6dfa0bffdda admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:45.485989 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0daabe-dc71-42f2-9710-c364d5619fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:45.486795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0daabe-dc71-42f2-9710-c364d5619fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:45.487606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0daabe-dc71-42f2-9710-c364d5619fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:45.487889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0daabe-dc71-42f2-9710-c364d5619fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:45.498358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0daabe-dc71-42f2-9710-c364d5619fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:45.499979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0daabe-dc71-42f2-9710-c364d5619fa7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:45.511724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1195/2388] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:00:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:00:49.668564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546d9e3-04df-41e2-b46d-be9dbaecd467 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:49.669369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546d9e3-04df-41e2-b46d-be9dbaecd467 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:49.669369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546d9e3-04df-41e2-b46d-be9dbaecd467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:49.690543 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c546d9e3-04df-41e2-b46d-be9dbaecd467 None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:49.691713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c546d9e3-04df-41e2-b46d-be9dbaecd467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:49.776353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1194/2389] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 05:00:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:49.790714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0daabe-dc71-42f2-9710-c364d5619fa7 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:49.793511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec0daabe-dc71-42f2-9710-c364d5619fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:49.927657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0daabe-dc71-42f2-9710-c364d5619fa7 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:49.977868 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ddfb9-7741-4059-9404-921d5d10084f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:49.978553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ddfb9-7741-4059-9404-921d5d10084f service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:49.978836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ddfb9-7741-4059-9404-921d5d10084f service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:49.979114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ddfb9-7741-4059-9404-921d5d10084f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:49.995657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ddfb9-7741-4059-9404-921d5d10084f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:49.997057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ddfb9-7741-4059-9404-921d5d10084f service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:50.021572 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1196/2390] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:00:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 05:00:53.386664 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb026e-4365-407b-a338-c527fcd312fd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:53.387540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb026e-4365-407b-a338-c527fcd312fd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:53.387980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb026e-4365-407b-a338-c527fcd312fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:53.447767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f0fb026e-4365-407b-a338-c527fcd312fd None None] MFA Rules not processed for user `407ad282edd74fb1adf62fb63306779c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:00:53.449251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f0fb026e-4365-407b-a338-c527fcd312fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:00:53.518772 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:53.646865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1195/2391] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:00:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:00:53.657453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ddfb9-7741-4059-9404-921d5d10084f service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:53.673821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:00:53.696198 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ddfb9-7741-4059-9404-921d5d10084f service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:53.708890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23b7ae-9f4d-4953-acb8-08e810bffb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:53.710002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23b7ae-9f4d-4953-acb8-08e810bffb7c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:53.710575 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23b7ae-9f4d-4953-acb8-08e810bffb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:53.711346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23b7ae-9f4d-4953-acb8-08e810bffb7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:53.712525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23b7ae-9f4d-4953-acb8-08e810bffb7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:53.720040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23b7ae-9f4d-4953-acb8-08e810bffb7c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:53.785871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1197/2392] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:00:53 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:53.789608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb026e-4365-407b-a338-c527fcd312fd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:53.809431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb026e-4365-407b-a338-c527fcd312fd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:53.969547 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91ecc0-0090-4465-90f7-8a2dc4c3b1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:53.970275 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91ecc0-0090-4465-90f7-8a2dc4c3b1cd admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:53.970661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91ecc0-0090-4465-90f7-8a2dc4c3b1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:53.971671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91ecc0-0090-4465-90f7-8a2dc4c3b1cd admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:53.972747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91ecc0-0090-4465-90f7-8a2dc4c3b1cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:53.974856 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91ecc0-0090-4465-90f7-8a2dc4c3b1cd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:54.034778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1196/2393] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:00:53 2022] POST /identity/v3/users => generated 296 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:00:54.038518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23b7ae-9f4d-4953-acb8-08e810bffb7c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:54.053038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23b7ae-9f4d-4953-acb8-08e810bffb7c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:54.065579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c6a19-df43-467b-89cc-02773ef6c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:54.066476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c6a19-df43-467b-89cc-02773ef6c09b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:54.066839 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c6a19-df43-467b-89cc-02773ef6c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:54.067638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c6a19-df43-467b-89cc-02773ef6c09b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:54.068029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c6a19-df43-467b-89cc-02773ef6c09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:54.069921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c6a19-df43-467b-89cc-02773ef6c09b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:54.105675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1198/2394] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:54.106633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91ecc0-0090-4465-90f7-8a2dc4c3b1cd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:54.126975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91ecc0-0090-4465-90f7-8a2dc4c3b1cd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:54.134102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8ebc5-3ca7-4fbf-a354-b4b692ccf053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:54.134936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8ebc5-3ca7-4fbf-a354-b4b692ccf053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:54.135223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8ebc5-3ca7-4fbf-a354-b4b692ccf053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:54.135501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8ebc5-3ca7-4fbf-a354-b4b692ccf053 admin admin] PATH_INFO: `/v3/projects/045ba324caf94e62a62b3ed245adf7b6/users/7c996bb1073f43fc9c7ce5509f2c4cc1/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:54.155443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8ebc5-3ca7-4fbf-a354-b4b692ccf053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=045ba324caf94e62a62b3ed245adf7b6, user_id=7c996bb1073f43fc9c7ce5509f2c4cc1, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:54.161961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8ebc5-3ca7-4fbf-a354-b4b692ccf053 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:54.197831 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1197/2395] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:54 2022] PUT /identity/v3/projects/045ba324caf94e62a62b3ed245adf7b6/users/7c996bb1073f43fc9c7ce5509f2c4cc1/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:00:54.199252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c6a19-df43-467b-89cc-02773ef6c09b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:54.218638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c6a19-df43-467b-89cc-02773ef6c09b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:54.334932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb7f17-9998-46ce-976e-2e82500db4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:54.334932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb7f17-9998-46ce-976e-2e82500db4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:54.335288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb7f17-9998-46ce-976e-2e82500db4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:54.335397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb7f17-9998-46ce-976e-2e82500db4c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:54.336436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb7f17-9998-46ce-976e-2e82500db4c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:54.338200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb7f17-9998-46ce-976e-2e82500db4c0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:54.391898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1199/2396] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:00:54.393682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8ebc5-3ca7-4fbf-a354-b4b692ccf053 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:00:54.423228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8ebc5-3ca7-4fbf-a354-b4b692ccf053 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:00:54.464200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f11821-046a-444e-a820-17440b349d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:00:54.464896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f11821-046a-444e-a820-17440b349d23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:00:54.465369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f11821-046a-444e-a820-17440b349d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:00:54.465819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f11821-046a-444e-a820-17440b349d23 admin admin] PATH_INFO: `/v3/projects/045ba324caf94e62a62b3ed245adf7b6/users/7c996bb1073f43fc9c7ce5509f2c4cc1/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:00:54.468321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f11821-046a-444e-a820-17440b349d23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=045ba324caf94e62a62b3ed245adf7b6, user_id=7c996bb1073f43fc9c7ce5509f2c4cc1, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:00:54.476201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f11821-046a-444e-a820-17440b349d23 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:00:54.515933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1198/2397] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:00:54 2022] PUT /identity/v3/projects/045ba324caf94e62a62b3ed245adf7b6/users/7c996bb1073f43fc9c7ce5509f2c4cc1/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:01:02.211912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dff09-c942-44c5-b36b-ae0d7b3da983 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:02.212483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dff09-c942-44c5-b36b-ae0d7b3da983 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:02.212927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dff09-c942-44c5-b36b-ae0d7b3da983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:02.243759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-275dff09-c942-44c5-b36b-ae0d7b3da983 None None] MFA Rules not processed for user `b98e41589d15488380a43000a82c2b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:01:02.247227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-275dff09-c942-44c5-b36b-ae0d7b3da983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:02.391849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1200/2398] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:01:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:01:04.500028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:01:05.458010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df225b-6dca-4e8f-a850-d59d099b61d9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:05.458259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df225b-6dca-4e8f-a850-d59d099b61d9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:05.458387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df225b-6dca-4e8f-a850-d59d099b61d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:05.526874 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a6df225b-6dca-4e8f-a850-d59d099b61d9 None None] MFA Rules not processed for user `7c996bb1073f43fc9c7ce5509f2c4cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:01:05.528210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a6df225b-6dca-4e8f-a850-d59d099b61d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:05.668985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1199/2399] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:01:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:01:05.683876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dff09-c942-44c5-b36b-ae0d7b3da983 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:01:05.685395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-275dff09-c942-44c5-b36b-ae0d7b3da983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:05.953131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dff09-c942-44c5-b36b-ae0d7b3da983 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:01:05.964901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0572d-42ea-478c-9673-8b01fc18aedb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:01:05.965957 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0572d-42ea-478c-9673-8b01fc18aedb service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:05.966554 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0572d-42ea-478c-9673-8b01fc18aedb service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:05.967026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0572d-42ea-478c-9673-8b01fc18aedb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:05.977444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:01:06.139205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0572d-42ea-478c-9673-8b01fc18aedb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:01:06.140863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0572d-42ea-478c-9673-8b01fc18aedb service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:01:06.162450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1201/2400] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:01:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:01:15.662667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=34 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:01:16.147500 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=24 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:01:23.786128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df225b-6dca-4e8f-a850-d59d099b61d9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:01:23.788368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a6df225b-6dca-4e8f-a850-d59d099b61d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:23.864004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df225b-6dca-4e8f-a850-d59d099b61d9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:01:23.869629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaf328-f70b-4e54-9905-8d4537fc2fd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:01:23.870075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaf328-f70b-4e54-9905-8d4537fc2fd7 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:23.870359 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaf328-f70b-4e54-9905-8d4537fc2fd7 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:23.870661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaf328-f70b-4e54-9905-8d4537fc2fd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:23.872537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bbdaf328-f70b-4e54-9905-8d4537fc2fd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:23.884878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaf328-f70b-4e54-9905-8d4537fc2fd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:01:23.886480 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaf328-f70b-4e54-9905-8d4537fc2fd7 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:01:23.897106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1200/2401] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:01:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:01:24.891652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76d1ec-be70-4717-be19-91c183c36d97 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:24.892174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76d1ec-be70-4717-be19-91c183c36d97 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:24.892257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76d1ec-be70-4717-be19-91c183c36d97 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:24.894337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1202/2402] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 05:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 05:01:24.900007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07383483-d637-47d6-ae12-c8e9bab9c1b1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:24.900464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07383483-d637-47d6-ae12-c8e9bab9c1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:24.900573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07383483-d637-47d6-ae12-c8e9bab9c1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:24.921129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-07383483-d637-47d6-ae12-c8e9bab9c1b1 None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:01:24.922347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-07383483-d637-47d6-ae12-c8e9bab9c1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:24.933009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1201/2403] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 05:01:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:01:24.946131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76d1ec-be70-4717-be19-91c183c36d97 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:01:24.949081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b76d1ec-be70-4717-be19-91c183c36d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:25.020149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76d1ec-be70-4717-be19-91c183c36d97 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:01:25.023870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38d512-1823-4d05-8eca-cca7b66cf09d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '548acf58bf9140ec9f1ba126e10deb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:01:25.024511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38d512-1823-4d05-8eca-cca7b66cf09d service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:25.024796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38d512-1823-4d05-8eca-cca7b66cf09d service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:25.025076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38d512-1823-4d05-8eca-cca7b66cf09d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:25.036614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38d512-1823-4d05-8eca-cca7b66cf09d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:01:25.038065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38d512-1823-4d05-8eca-cca7b66cf09d service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:01:25.053840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1203/2404] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:01:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:01:25.123224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4793446-b4b0-4c91-b26c-cb4bad98e208 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:25.123224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4793446-b4b0-4c91-b26c-cb4bad98e208 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:25.123224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4793446-b4b0-4c91-b26c-cb4bad98e208 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:25.124819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1202/2405] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 05:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 05:01:25.130046 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef525dc8-36b0-45b8-926b-78f75c03513b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:25.130361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef525dc8-36b0-45b8-926b-78f75c03513b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:25.130661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef525dc8-36b0-45b8-926b-78f75c03513b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:25.151387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ef525dc8-36b0-45b8-926b-78f75c03513b None None] MFA Rules not processed for user `426be56424504e958402d785a068d2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:01:25.152614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ef525dc8-36b0-45b8-926b-78f75c03513b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:25.203516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1204/2406] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 05:01:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:01:25.215902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4793446-b4b0-4c91-b26c-cb4bad98e208 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:01:25.217592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b4793446-b4b0-4c91-b26c-cb4bad98e208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:25.230694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4793446-b4b0-4c91-b26c-cb4bad98e208 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:01:25.234164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8227d0-f5f6-4a41-91b6-a53b7d375c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:01:25.234791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8227d0-f5f6-4a41-91b6-a53b7d375c99 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:25.235063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8227d0-f5f6-4a41-91b6-a53b7d375c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:25.235332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8227d0-f5f6-4a41-91b6-a53b7d375c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:25.242943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8227d0-f5f6-4a41-91b6-a53b7d375c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:01:25.244353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8227d0-f5f6-4a41-91b6-a53b7d375c99 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:01:25.251903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1203/2407] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:01:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:01:25.337723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f32bd1-b0a9-497e-919e-b086c1e34de1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:25.338018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f32bd1-b0a9-497e-919e-b086c1e34de1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:25.338299 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f32bd1-b0a9-497e-919e-b086c1e34de1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:25.339929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1205/2408] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 05:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 05:01:25.344617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53b70-97d2-42b4-b0da-e80ff97d668e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:25.344906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53b70-97d2-42b4-b0da-e80ff97d668e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:25.345189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53b70-97d2-42b4-b0da-e80ff97d668e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:25.366341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e7b53b70-97d2-42b4-b0da-e80ff97d668e None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:01:25.367575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e7b53b70-97d2-42b4-b0da-e80ff97d668e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:25.377095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1204/2409] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 05:01:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:01:25.390559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f32bd1-b0a9-497e-919e-b086c1e34de1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:01:25.402091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f32bd1-b0a9-497e-919e-b086c1e34de1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:01:25.405918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331463f-7e44-4673-ad68-9232377a1457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:01:25.406571 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331463f-7e44-4673-ad68-9232377a1457 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:25.406898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331463f-7e44-4673-ad68-9232377a1457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:25.407177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331463f-7e44-4673-ad68-9232377a1457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:25.417781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331463f-7e44-4673-ad68-9232377a1457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:01:25.419256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331463f-7e44-4673-ad68-9232377a1457 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:01:25.430774 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1206/2410] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:01:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:01:31.187459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b4fe5-03a2-4148-b2b6-d7a7a8551acd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:31.187948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b4fe5-03a2-4148-b2b6-d7a7a8551acd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:31.188061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b4fe5-03a2-4148-b2b6-d7a7a8551acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:31.208008 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d86b4fe5-03a2-4148-b2b6-d7a7a8551acd None None] MFA Rules not processed for user `407ad282edd74fb1adf62fb63306779c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:01:31.209205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d86b4fe5-03a2-4148-b2b6-d7a7a8551acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:31.216011 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1205/2411] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:01:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:01:34.275276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a39996-5e9e-4ea0-a83f-479bd280570f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:34.275698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a39996-5e9e-4ea0-a83f-479bd280570f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:34.275935 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a39996-5e9e-4ea0-a83f-479bd280570f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:34.303066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-88a39996-5e9e-4ea0-a83f-479bd280570f None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:01:34.304802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-88a39996-5e9e-4ea0-a83f-479bd280570f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:34.362550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1207/2412] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 05:01:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:01:34.383730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b4fe5-03a2-4148-b2b6-d7a7a8551acd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:01:34.450932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b4fe5-03a2-4148-b2b6-d7a7a8551acd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:01:34.461883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37490642-6a91-4090-9640-eea875a9dfb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:01:34.462544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37490642-6a91-4090-9640-eea875a9dfb2 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:34.462825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37490642-6a91-4090-9640-eea875a9dfb2 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:34.463096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37490642-6a91-4090-9640-eea875a9dfb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:34.472203 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37490642-6a91-4090-9640-eea875a9dfb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:01:34.473789 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37490642-6a91-4090-9640-eea875a9dfb2 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:01:34.481916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1206/2413] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:01:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 05:01:44.354959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=80 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:01:44.478665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=79 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:01:52.841248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc3ad2-f5c0-4b1a-8a9a-e40c2f3cb226 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:52.841807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc3ad2-f5c0-4b1a-8a9a-e40c2f3cb226 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:52.841807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc3ad2-f5c0-4b1a-8a9a-e40c2f3cb226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:52.861230 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-93dc3ad2-f5c0-4b1a-8a9a-e40c2f3cb226 None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:01:52.862463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-93dc3ad2-f5c0-4b1a-8a9a-e40c2f3cb226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:01:52.870166 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1208/2414] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 05:01:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:01:52.882912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37490642-6a91-4090-9640-eea875a9dfb2 service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:01:52.893442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37490642-6a91-4090-9640-eea875a9dfb2 service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:01:52.899460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde139e5-8307-4868-a70e-cc490c0cd6c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:01:52.900068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde139e5-8307-4868-a70e-cc490c0cd6c5 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:01:52.900348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde139e5-8307-4868-a70e-cc490c0cd6c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:01:52.900619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde139e5-8307-4868-a70e-cc490c0cd6c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:01:52.908747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde139e5-8307-4868-a70e-cc490c0cd6c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:01:52.910918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde139e5-8307-4868-a70e-cc490c0cd6c5 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:01:52.919431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1207/2415] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:01:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 05:02:02.856031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:02:02.916403 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:02:07.097549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab6727-945b-4774-ac42-cde6bc41a4ff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:07.098755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab6727-945b-4774-ac42-cde6bc41a4ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:07.099338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab6727-945b-4774-ac42-cde6bc41a4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:07.126909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-58ab6727-945b-4774-ac42-cde6bc41a4ff None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:02:07.130058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-58ab6727-945b-4774-ac42-cde6bc41a4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:07.148418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1209/2416] 10.222.0.20 () {58 vars in 967 bytes} [Sat Jul 30 05:02:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:02:07.158156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde139e5-8307-4868-a70e-cc490c0cd6c5 service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:07.170936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde139e5-8307-4868-a70e-cc490c0cd6c5 service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:07.177189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad960ae-f496-4c63-b31c-a6374f181f78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:07.177848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad960ae-f496-4c63-b31c-a6374f181f78 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:07.178344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad960ae-f496-4c63-b31c-a6374f181f78 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:07.178683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad960ae-f496-4c63-b31c-a6374f181f78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:07.188381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad960ae-f496-4c63-b31c-a6374f181f78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:07.189951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad960ae-f496-4c63-b31c-a6374f181f78 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:07.200340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1208/2417] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:02:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 30 05:02:07.689514 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341952e8-258b-4abc-af2b-3335af9c7ac9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:07.689721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341952e8-258b-4abc-af2b-3335af9c7ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:07.689985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341952e8-258b-4abc-af2b-3335af9c7ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:07.716297 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-341952e8-258b-4abc-af2b-3335af9c7ac9 None None] MFA Rules not processed for user `334f2d8d9d9a4f7a90586999398f8a40`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:02:07.718064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-341952e8-258b-4abc-af2b-3335af9c7ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:07.804381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1210/2418] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:02:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:02:10.972866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f528b-3b81-41b4-a506-9c8c568e4683 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:10.973339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f528b-3b81-41b4-a506-9c8c568e4683 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:10.973439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f528b-3b81-41b4-a506-9c8c568e4683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:10.992374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ff5f528b-3b81-41b4-a506-9c8c568e4683 None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:02:10.993596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ff5f528b-3b81-41b4-a506-9c8c568e4683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:11.000654 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1209/2419] 10.222.0.20 () {58 vars in 967 bytes} [Sat Jul 30 05:02:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:02:11.017008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341952e8-258b-4abc-af2b-3335af9c7ac9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:11.029779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341952e8-258b-4abc-af2b-3335af9c7ac9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:11.041232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e098dd4-3815-4ee0-a539-076d7888ab9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:11.041863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e098dd4-3815-4ee0-a539-076d7888ab9e service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:11.042147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e098dd4-3815-4ee0-a539-076d7888ab9e service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:11.042420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e098dd4-3815-4ee0-a539-076d7888ab9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:11.051877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e098dd4-3815-4ee0-a539-076d7888ab9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:11.053469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e098dd4-3815-4ee0-a539-076d7888ab9e service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:11.061544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1211/2420] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:02:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 05:02:11.755737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f528b-3b81-41b4-a506-9c8c568e4683 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:11.772442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f528b-3b81-41b4-a506-9c8c568e4683 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:11.777757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57496944-65eb-49c7-9a84-78df38950051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:11.779339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57496944-65eb-49c7-9a84-78df38950051 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:11.779847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57496944-65eb-49c7-9a84-78df38950051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:11.780476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57496944-65eb-49c7-9a84-78df38950051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:11.824243 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57496944-65eb-49c7-9a84-78df38950051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:11.827298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57496944-65eb-49c7-9a84-78df38950051 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:11.841657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1210/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:02:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:02:13.819286 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e098dd4-3815-4ee0-a539-076d7888ab9e service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:13.821356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e098dd4-3815-4ee0-a539-076d7888ab9e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:13.900529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e098dd4-3815-4ee0-a539-076d7888ab9e service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:13.905112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:13.905762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:13.906044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:13.906319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] PATH_INFO: `/v3/users/b98e41589d15488380a43000a82c2b3c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:13.907640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b98e41589d15488380a43000a82c2b3c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:13.910055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:13.961640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b98e41589d15488380a43000a82c2b3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:13.968972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b98e41589d15488380a43000a82c2b3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:13.969648 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b98e41589d15488380a43000a82c2b3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:13.970370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b98e41589d15488380a43000a82c2b3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:13.987849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-11fd78bf-2464-4daa-b2bb-6235168dad87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b98e41589d15488380a43000a82c2b3c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:14.002098 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1212/2422] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:02:13 2022] DELETE /identity/v3/users/b98e41589d15488380a43000a82c2b3c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:02:14.189794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57496944-65eb-49c7-9a84-78df38950051 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:14.201555 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57496944-65eb-49c7-9a84-78df38950051 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:14.269576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:14.270006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.270342 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.270650 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] PATH_INFO: `/v3/projects/f102af4a74e946caa9fefc7ad0ae148c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.272120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f102af4a74e946caa9fefc7ad0ae148c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:14.273962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:14.331548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f102af4a74e946caa9fefc7ad0ae148c', 'request_id': 'req-0f89e1de-3845-4280-b4eb-bf0f0aafd027'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:14.340723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f102af4a74e946caa9fefc7ad0ae148c', 'request_id': 'req-0f89e1de-3845-4280-b4eb-bf0f0aafd027'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:14.348434 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f102af4a74e946caa9fefc7ad0ae148c', 'request_id': 'req-0f89e1de-3845-4280-b4eb-bf0f0aafd027'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:14.360751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] The token cache is being invalidate because project f102af4a74e946caa9fefc7ad0ae148c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:02:14.361244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:14.362064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0f89e1de-3845-4280-b4eb-bf0f0aafd027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:14.364194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1211/2423] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:02:14 2022] DELETE /identity/v3/projects/f102af4a74e946caa9fefc7ad0ae148c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:02:14.369050 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e310509-6fa3-4afd-b6f8-a11f188bb0ab None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.369339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e310509-6fa3-4afd-b6f8-a11f188bb0ab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.369623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e310509-6fa3-4afd-b6f8-a11f188bb0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.378873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:02:14.391086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6e310509-6fa3-4afd-b6f8-a11f188bb0ab None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:02:14.392484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6e310509-6fa3-4afd-b6f8-a11f188bb0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:14.462385 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1213/2424] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:02:14.470714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755d9ce-ed13-4ac2-9dba-90b0c01c5eef None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.470714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755d9ce-ed13-4ac2-9dba-90b0c01c5eef None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.470885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755d9ce-ed13-4ac2-9dba-90b0c01c5eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.501880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6755d9ce-ed13-4ac2-9dba-90b0c01c5eef None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:02:14.503958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6755d9ce-ed13-4ac2-9dba-90b0c01c5eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:14.516901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1212/2425] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:02:14.529509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e310509-6fa3-4afd-b6f8-a11f188bb0ab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:14.541529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e310509-6fa3-4afd-b6f8-a11f188bb0ab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:14.548840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459826fc-0ab1-4c23-b067-899c835411cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:14.550573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459826fc-0ab1-4c23-b067-899c835411cf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.551074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459826fc-0ab1-4c23-b067-899c835411cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.551527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459826fc-0ab1-4c23-b067-899c835411cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.552606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459826fc-0ab1-4c23-b067-899c835411cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:02:14.553166 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459826fc-0ab1-4c23-b067-899c835411cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:14.555173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459826fc-0ab1-4c23-b067-899c835411cf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:14.578082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1214/2426] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:02:14.579647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755d9ce-ed13-4ac2-9dba-90b0c01c5eef None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:14.590034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755d9ce-ed13-4ac2-9dba-90b0c01c5eef None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:14.595146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ae5aa-55b6-410f-ab02-c5640f92a558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:14.595778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ae5aa-55b6-410f-ab02-c5640f92a558 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.596075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ae5aa-55b6-410f-ab02-c5640f92a558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.596354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ae5aa-55b6-410f-ab02-c5640f92a558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.598056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ae5aa-55b6-410f-ab02-c5640f92a558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:14.599695 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ae5aa-55b6-410f-ab02-c5640f92a558 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:14.638382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1213/2427] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:02:14 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:02:14.639545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459826fc-0ab1-4c23-b067-899c835411cf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:14.654969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459826fc-0ab1-4c23-b067-899c835411cf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:14.734296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004ce1a-ff83-4ccc-8ffc-683dff088078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:14.734974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004ce1a-ff83-4ccc-8ffc-683dff088078 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.735845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004ce1a-ff83-4ccc-8ffc-683dff088078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.736135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004ce1a-ff83-4ccc-8ffc-683dff088078 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.737726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004ce1a-ff83-4ccc-8ffc-683dff088078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:14.740090 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004ce1a-ff83-4ccc-8ffc-683dff088078 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:14.767088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1215/2428] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:02:14 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:02:14.771356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ae5aa-55b6-410f-ab02-c5640f92a558 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:14.782152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ae5aa-55b6-410f-ab02-c5640f92a558 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:14.787358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1082f-ae53-4f67-b595-16f5c9b418d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:14.788137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1082f-ae53-4f67-b595-16f5c9b418d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.788419 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1082f-ae53-4f67-b595-16f5c9b418d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.788693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1082f-ae53-4f67-b595-16f5c9b418d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.789396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1082f-ae53-4f67-b595-16f5c9b418d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:14.791180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1082f-ae53-4f67-b595-16f5c9b418d6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:14.803271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1214/2429] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:02:14.807133 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004ce1a-ff83-4ccc-8ffc-683dff088078 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:14.825301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004ce1a-ff83-4ccc-8ffc-683dff088078 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:14.831465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21cfe2-c617-4120-a1e3-27fd9b6d3426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:14.832248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21cfe2-c617-4120-a1e3-27fd9b6d3426 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.832764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21cfe2-c617-4120-a1e3-27fd9b6d3426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.833140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21cfe2-c617-4120-a1e3-27fd9b6d3426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.834079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21cfe2-c617-4120-a1e3-27fd9b6d3426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:14.837685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c21cfe2-c617-4120-a1e3-27fd9b6d3426 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:14.860206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1082f-ae53-4f67-b595-16f5c9b418d6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:14.860403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1216/2430] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:02:14.873880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1082f-ae53-4f67-b595-16f5c9b418d6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:14.883173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d0523-68ca-4a7b-9cd2-569583b1c0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:14.884040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d0523-68ca-4a7b-9cd2-569583b1c0ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.884319 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d0523-68ca-4a7b-9cd2-569583b1c0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.884594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d0523-68ca-4a7b-9cd2-569583b1c0ab admin admin] PATH_INFO: `/v3/projects/f3675768b09147eeaacfe0b36809609a/users/c82980bcc812465d9f9026a53e639ec2/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.901279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d0523-68ca-4a7b-9cd2-569583b1c0ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3675768b09147eeaacfe0b36809609a, user_id=c82980bcc812465d9f9026a53e639ec2, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:14.904242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d0523-68ca-4a7b-9cd2-569583b1c0ab admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:14.906728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:02:14.928579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1215/2431] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:02:14 2022] PUT /identity/v3/projects/f3675768b09147eeaacfe0b36809609a/users/c82980bcc812465d9f9026a53e639ec2/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:02:14.939558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac2a96-b412-411a-95d5-f6bae06f8ba7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:14.939851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac2a96-b412-411a-95d5-f6bae06f8ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:14.940136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac2a96-b412-411a-95d5-f6bae06f8ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:14.981202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-68ac2a96-b412-411a-95d5-f6bae06f8ba7 None None] MFA Rules not processed for user `c82980bcc812465d9f9026a53e639ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:02:14.982790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-68ac2a96-b412-411a-95d5-f6bae06f8ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:15.069950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:02:15.075077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1217/2432] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:02:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:02:15.088148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d0523-68ca-4a7b-9cd2-569583b1c0ab admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:15.090497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8f0d0523-68ca-4a7b-9cd2-569583b1c0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:15.165432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d0523-68ca-4a7b-9cd2-569583b1c0ab admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:15.169516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db81061-ea70-4cee-88d6-57ef1f2d350d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:15.170663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db81061-ea70-4cee-88d6-57ef1f2d350d service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:15.171055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db81061-ea70-4cee-88d6-57ef1f2d350d service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:15.171582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db81061-ea70-4cee-88d6-57ef1f2d350d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:15.183549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db81061-ea70-4cee-88d6-57ef1f2d350d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:15.185681 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db81061-ea70-4cee-88d6-57ef1f2d350d service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:15.199877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1216/2433] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:02:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:02:19.588115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eed9b0-3f8c-458a-88be-b0e686df2117 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:19.588568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eed9b0-3f8c-458a-88be-b0e686df2117 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:19.588790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eed9b0-3f8c-458a-88be-b0e686df2117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:19.609043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e3eed9b0-3f8c-458a-88be-b0e686df2117 None None] MFA Rules not processed for user `a8abb58f9c274e568d2d0fe3d4df11de`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:02:19.610342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e3eed9b0-3f8c-458a-88be-b0e686df2117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:19.669042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1218/2434] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:02:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:02:25.193532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:02:26.017610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db81061-ea70-4cee-88d6-57ef1f2d350d service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:26.085563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db81061-ea70-4cee-88d6-57ef1f2d350d service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:26.090159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:26.090857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:26.091141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:26.091414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] PATH_INFO: `/v3/users/c82980bcc812465d9f9026a53e639ec2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:26.092594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c82980bcc812465d9f9026a53e639ec2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:26.094545 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:26.143698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c82980bcc812465d9f9026a53e639ec2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.150540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c82980bcc812465d9f9026a53e639ec2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.151214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c82980bcc812465d9f9026a53e639ec2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.151813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c82980bcc812465d9f9026a53e639ec2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.165165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c82980bcc812465d9f9026a53e639ec2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.179527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1217/2435] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:02:26 2022] DELETE /identity/v3/users/c82980bcc812465d9f9026a53e639ec2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:02:26.183953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e332834-eda9-42e9-bebf-1eab58a9d6c7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:26.184508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e332834-eda9-42e9-bebf-1eab58a9d6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:26.184590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e332834-eda9-42e9-bebf-1eab58a9d6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:26.203434 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3e332834-eda9-42e9-bebf-1eab58a9d6c7 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:02:26.204636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e332834-eda9-42e9-bebf-1eab58a9d6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:26.262389 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1219/2436] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:02:26.272632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:26.274059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:26.333041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb31ad-3949-4018-9ed8-0797d82c849e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:26.336776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264b7a5-b824-441f-9040-24f9ff8c85af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:26.337316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264b7a5-b824-441f-9040-24f9ff8c85af service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:26.337590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264b7a5-b824-441f-9040-24f9ff8c85af service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:26.337857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264b7a5-b824-441f-9040-24f9ff8c85af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:26.346778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264b7a5-b824-441f-9040-24f9ff8c85af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:26.348299 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264b7a5-b824-441f-9040-24f9ff8c85af service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:26.357515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1218/2437] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:02:26.619452 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e332834-eda9-42e9-bebf-1eab58a9d6c7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:02:26.620887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e332834-eda9-42e9-bebf-1eab58a9d6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:26.633539 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e332834-eda9-42e9-bebf-1eab58a9d6c7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:02:26.638604 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:02:26.639259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:26.639601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:26.639803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] PATH_INFO: `/v3/projects/f3675768b09147eeaacfe0b36809609a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:26.641277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3675768b09147eeaacfe0b36809609a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:02:26.643889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:02:26.695614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3675768b09147eeaacfe0b36809609a', 'request_id': 'req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.703840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3675768b09147eeaacfe0b36809609a', 'request_id': 'req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.710629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3675768b09147eeaacfe0b36809609a', 'request_id': 'req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.722898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] The token cache is being invalidate because project f3675768b09147eeaacfe0b36809609a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:02:26.723899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.724595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a9bdcb27-1d42-4521-bcd7-f2aa90498dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:02:26.726250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1220/2438] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:02:26 2022] DELETE /identity/v3/projects/f3675768b09147eeaacfe0b36809609a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:02:31.562354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f339d5e-0512-45c5-9a40-231e898b23d8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:02:31.562892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f339d5e-0512-45c5-9a40-231e898b23d8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:02:31.563022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f339d5e-0512-45c5-9a40-231e898b23d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:02:31.581973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1f339d5e-0512-45c5-9a40-231e898b23d8 None None] MFA Rules not processed for user `25db6f7402ae4a14a04fab0a9915ab77`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:02:31.583669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1f339d5e-0512-45c5-9a40-231e898b23d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:02:31.600109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:02:31.640220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1219/2439] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:02:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:02:36.719174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:02:41.633630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=15 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:07.028866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6a9b0-37f9-4184-8452-34d28c1046b2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:07.029414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6a9b0-37f9-4184-8452-34d28c1046b2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:07.029414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6a9b0-37f9-4184-8452-34d28c1046b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:07.049765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a4a6a9b0-37f9-4184-8452-34d28c1046b2 None None] MFA Rules not processed for user `d3a56c6d44b74429a6616cdfc2869b71`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:07.051285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a4a6a9b0-37f9-4184-8452-34d28c1046b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:07.117342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1221/2440] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:03:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:09.093740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f339d5e-0512-45c5-9a40-231e898b23d8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:09.095281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1f339d5e-0512-45c5-9a40-231e898b23d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:09.154158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f339d5e-0512-45c5-9a40-231e898b23d8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:09.157724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21860f-02e0-40de-9f07-1fb215a0661d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:09.158337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21860f-02e0-40de-9f07-1fb215a0661d service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:09.158659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21860f-02e0-40de-9f07-1fb215a0661d service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:09.159112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21860f-02e0-40de-9f07-1fb215a0661d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:09.161304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fb21860f-02e0-40de-9f07-1fb215a0661d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:09.238563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21860f-02e0-40de-9f07-1fb215a0661d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:09.240270 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21860f-02e0-40de-9f07-1fb215a0661d service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:09.251897 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1220/2441] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:03:11.216042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6a9b0-37f9-4184-8452-34d28c1046b2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:11.217510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a4a6a9b0-37f9-4184-8452-34d28c1046b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:11.283306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6a9b0-37f9-4184-8452-34d28c1046b2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:11.288611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:11.289306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:11.289583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:11.289853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] PATH_INFO: `/v3/users/334f2d8d9d9a4f7a90586999398f8a40` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:11.291022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=334f2d8d9d9a4f7a90586999398f8a40)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:11.292936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:11.340384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334f2d8d9d9a4f7a90586999398f8a40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.347173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334f2d8d9d9a4f7a90586999398f8a40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.347848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334f2d8d9d9a4f7a90586999398f8a40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.348445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334f2d8d9d9a4f7a90586999398f8a40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.360121 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334f2d8d9d9a4f7a90586999398f8a40'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.372513 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1222/2442] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:03:11 2022] DELETE /identity/v3/users/334f2d8d9d9a4f7a90586999398f8a40 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:03:11.547660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21860f-02e0-40de-9f07-1fb215a0661d service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:11.562451 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21860f-02e0-40de-9f07-1fb215a0661d service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:11.621475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:11.622112 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:11.622387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:11.622713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] PATH_INFO: `/v3/projects/a6a02f6df4ed4365b3ce1fa970f9cf41` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:11.623926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a02f6df4ed4365b3ce1fa970f9cf41)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:11.625703 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:11.661047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:11.662508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:11.671570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:11.676741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a02f6df4ed4365b3ce1fa970f9cf41', 'request_id': 'req-4bf24015-d945-47c6-b986-5f6367f17d0b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.685013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a02f6df4ed4365b3ce1fa970f9cf41', 'request_id': 'req-4bf24015-d945-47c6-b986-5f6367f17d0b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.691546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a02f6df4ed4365b3ce1fa970f9cf41', 'request_id': 'req-4bf24015-d945-47c6-b986-5f6367f17d0b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.693152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:11.703369 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] The token cache is being invalidate because project a6a02f6df4ed4365b3ce1fa970f9cf41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:03:11.703653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.704329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4bf24015-d945-47c6-b986-5f6367f17d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:03:11.705880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1221/2443] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:03:11 2022] DELETE /identity/v3/projects/a6a02f6df4ed4365b3ce1fa970f9cf41 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:03:11.711448 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9972293-f243-4847-a6b7-585dd3e88842 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:11.711732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9972293-f243-4847-a6b7-585dd3e88842 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:11.712007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9972293-f243-4847-a6b7-585dd3e88842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:11.727826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c76c5-ffc6-495c-997c-dc0833a94ffe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:11.731086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c9972293-f243-4847-a6b7-585dd3e88842 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:11.732284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c9972293-f243-4847-a6b7-585dd3e88842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:11.779612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c28495-efff-449c-b88c-c7aa768a0919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:11.780233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c28495-efff-449c-b88c-c7aa768a0919 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:11.780510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c28495-efff-449c-b88c-c7aa768a0919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:11.780790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c28495-efff-449c-b88c-c7aa768a0919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:11.782612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-80c28495-efff-449c-b88c-c7aa768a0919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:11.794857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1222/2444] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:03:11.804396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065780e-4ad5-4d38-9b1d-9a0da73916b8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:11.804749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065780e-4ad5-4d38-9b1d-9a0da73916b8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:11.805017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065780e-4ad5-4d38-9b1d-9a0da73916b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:11.825632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5065780e-4ad5-4d38-9b1d-9a0da73916b8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:11.827264 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5065780e-4ad5-4d38-9b1d-9a0da73916b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:11.835398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1223/2445] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:11.840460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065780e-4ad5-4d38-9b1d-9a0da73916b8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:11.852741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065780e-4ad5-4d38-9b1d-9a0da73916b8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:11.857911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a93c2c-276a-4878-b070-856983904ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:11.858691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a93c2c-276a-4878-b070-856983904ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:11.859063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a93c2c-276a-4878-b070-856983904ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:11.859532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a93c2c-276a-4878-b070-856983904ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:11.860450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a93c2c-276a-4878-b070-856983904ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:03:11.860930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a93c2c-276a-4878-b070-856983904ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:11.863178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a93c2c-276a-4878-b070-856983904ce8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:11.875423 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c28495-efff-449c-b88c-c7aa768a0919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:11.877317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c28495-efff-449c-b88c-c7aa768a0919 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:11.878410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1224/2446] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:03:11.882798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a93c2c-276a-4878-b070-856983904ce8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:11.891117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1223/2447] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:03:11.893880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a93c2c-276a-4878-b070-856983904ce8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:11.899231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d93fae-6f2c-496f-a085-4d71fd46f4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:11.899936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d93fae-6f2c-496f-a085-4d71fd46f4aa admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:11.900221 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d93fae-6f2c-496f-a085-4d71fd46f4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:11.900527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d93fae-6f2c-496f-a085-4d71fd46f4aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:11.901194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d93fae-6f2c-496f-a085-4d71fd46f4aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:11.903172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d93fae-6f2c-496f-a085-4d71fd46f4aa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:11.932799 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1225/2448] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:03:11 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:03:11.937331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c28495-efff-449c-b88c-c7aa768a0919 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:11.949248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c28495-efff-449c-b88c-c7aa768a0919 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:12.014408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fb09d-d4c1-406d-85e2-3ed4156c775f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:12.015264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fb09d-d4c1-406d-85e2-3ed4156c775f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:12.015644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fb09d-d4c1-406d-85e2-3ed4156c775f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:12.016583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fb09d-d4c1-406d-85e2-3ed4156c775f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:12.016732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fb09d-d4c1-406d-85e2-3ed4156c775f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:12.018747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fb09d-d4c1-406d-85e2-3ed4156c775f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:12.044977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1224/2449] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:03:11 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:03:12.046615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d93fae-6f2c-496f-a085-4d71fd46f4aa admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:12.057763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d93fae-6f2c-496f-a085-4d71fd46f4aa admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:12.062470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16a856-e400-49af-9922-676768977a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:12.063133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16a856-e400-49af-9922-676768977a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:12.063526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16a856-e400-49af-9922-676768977a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:12.063719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16a856-e400-49af-9922-676768977a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:12.064424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16a856-e400-49af-9922-676768977a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:12.066075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16a856-e400-49af-9922-676768977a1a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:12.080530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1226/2450] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:03:12.081542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fb09d-d4c1-406d-85e2-3ed4156c775f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:12.092598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fb09d-d4c1-406d-85e2-3ed4156c775f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:12.098146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7045e-5111-44af-9f60-555e2a19c0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:12.098805 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7045e-5111-44af-9f60-555e2a19c0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:12.099083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7045e-5111-44af-9f60-555e2a19c0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:12.099351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7045e-5111-44af-9f60-555e2a19c0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:12.100366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7045e-5111-44af-9f60-555e2a19c0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:12.101970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7045e-5111-44af-9f60-555e2a19c0b7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:12.108067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:12.116437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1225/2451] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:03:12.120552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16a856-e400-49af-9922-676768977a1a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:12.131880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16a856-e400-49af-9922-676768977a1a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:12.136672 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff215a8-a85e-47a0-97b1-c4f72935c9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:12.137804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff215a8-a85e-47a0-97b1-c4f72935c9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:12.137804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff215a8-a85e-47a0-97b1-c4f72935c9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:12.138036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff215a8-a85e-47a0-97b1-c4f72935c9c6 admin admin] PATH_INFO: `/v3/projects/abbb209e2c194596bb3e8b9a6880646e/users/42d3708ddfcc491cb366054dcd5f8881/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:12.157229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff215a8-a85e-47a0-97b1-c4f72935c9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abbb209e2c194596bb3e8b9a6880646e, user_id=42d3708ddfcc491cb366054dcd5f8881, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:12.160497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff215a8-a85e-47a0-97b1-c4f72935c9c6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:12.181562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1227/2452] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:03:12 2022] PUT /identity/v3/projects/abbb209e2c194596bb3e8b9a6880646e/users/42d3708ddfcc491cb366054dcd5f8881/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:03:12.186786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5876d-79be-4fad-b92d-962d6c9b0eb9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:12.187078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5876d-79be-4fad-b92d-962d6c9b0eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:12.187351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5876d-79be-4fad-b92d-962d6c9b0eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:12.207965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dde5876d-79be-4fad-b92d-962d6c9b0eb9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:12.209210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dde5876d-79be-4fad-b92d-962d6c9b0eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:12.276182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1226/2453] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:03:12.286846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff215a8-a85e-47a0-97b1-c4f72935c9c6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:12.288559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dff215a8-a85e-47a0-97b1-c4f72935c9c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:12.348613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff215a8-a85e-47a0-97b1-c4f72935c9c6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:12.352272 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bf2e8-915a-4234-be71-4ac187e0a39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:12.352885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bf2e8-915a-4234-be71-4ac187e0a39e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:12.353157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bf2e8-915a-4234-be71-4ac187e0a39e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:12.353455 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bf2e8-915a-4234-be71-4ac187e0a39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:12.363849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bf2e8-915a-4234-be71-4ac187e0a39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:12.365273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bf2e8-915a-4234-be71-4ac187e0a39e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:12.369267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:12.376167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1228/2454] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:16.833859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6913bab-26a0-4459-b6a6-2b9abb99e434 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:16.834358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6913bab-26a0-4459-b6a6-2b9abb99e434 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:16.834499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6913bab-26a0-4459-b6a6-2b9abb99e434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:16.871075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e6913bab-26a0-4459-b6a6-2b9abb99e434 None None] MFA Rules not processed for user `42d3708ddfcc491cb366054dcd5f8881`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:16.872430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e6913bab-26a0-4459-b6a6-2b9abb99e434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:16.951436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1227/2455] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:03:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:16.956497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bf2e8-915a-4234-be71-4ac187e0a39e service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:16.966597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bf2e8-915a-4234-be71-4ac187e0a39e service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:16.971718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25a293-be40-4a84-9cbf-30775b0facf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:16.972824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25a293-be40-4a84-9cbf-30775b0facf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:16.973166 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25a293-be40-4a84-9cbf-30775b0facf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:16.973508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25a293-be40-4a84-9cbf-30775b0facf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:16.974274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25a293-be40-4a84-9cbf-30775b0facf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:16.976320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25a293-be40-4a84-9cbf-30775b0facf8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:17.010870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1229/2456] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:03:16 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:03:17.014834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6913bab-26a0-4459-b6a6-2b9abb99e434 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:17.028035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6913bab-26a0-4459-b6a6-2b9abb99e434 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:17.105451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b711-cb72-4010-8510-8fc559b25379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:17.107215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b711-cb72-4010-8510-8fc559b25379 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:17.107413 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b711-cb72-4010-8510-8fc559b25379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:17.107712 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b711-cb72-4010-8510-8fc559b25379 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:17.110573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8b711-cb72-4010-8510-8fc559b25379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:17.113329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8b711-cb72-4010-8510-8fc559b25379 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:17.139611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1228/2457] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:03:17 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:03:17.143686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25a293-be40-4a84-9cbf-30775b0facf8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:17.156135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25a293-be40-4a84-9cbf-30775b0facf8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:17.164097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0198e-75e3-4c31-9f54-97650b326e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:17.164097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0198e-75e3-4c31-9f54-97650b326e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:17.164097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0198e-75e3-4c31-9f54-97650b326e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:17.164097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0198e-75e3-4c31-9f54-97650b326e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:17.164579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0198e-75e3-4c31-9f54-97650b326e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:17.166356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0198e-75e3-4c31-9f54-97650b326e14 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:17.180563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1230/2458] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:03:17.185577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b711-cb72-4010-8510-8fc559b25379 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:17.198039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b711-cb72-4010-8510-8fc559b25379 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:17.207544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c9881-131c-4d72-a597-ac44acc24c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:17.209333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c9881-131c-4d72-a597-ac44acc24c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:17.210858 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c9881-131c-4d72-a597-ac44acc24c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:17.211287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c9881-131c-4d72-a597-ac44acc24c61 admin admin] PATH_INFO: `/v3/projects/a1b74e0af6924eaca6c3e94efb720a76/users/3b52445a83c44ed8ade18cfe23cdac14/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:17.233751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c9881-131c-4d72-a597-ac44acc24c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b74e0af6924eaca6c3e94efb720a76, user_id=3b52445a83c44ed8ade18cfe23cdac14, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:17.237496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c9881-131c-4d72-a597-ac44acc24c61 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:17.245078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:17.265593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1229/2459] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:03:17 2022] PUT /identity/v3/projects/a1b74e0af6924eaca6c3e94efb720a76/users/3b52445a83c44ed8ade18cfe23cdac14/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:03:17.269577 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0198e-75e3-4c31-9f54-97650b326e14 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:17.284614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0198e-75e3-4c31-9f54-97650b326e14 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:17.359759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bbbe0-8d45-4d1b-b6ed-31b51c9e13e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:17.360387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bbbe0-8d45-4d1b-b6ed-31b51c9e13e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:17.360666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bbbe0-8d45-4d1b-b6ed-31b51c9e13e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:17.360940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bbbe0-8d45-4d1b-b6ed-31b51c9e13e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:17.362186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bbbe0-8d45-4d1b-b6ed-31b51c9e13e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:17.364427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bbbe0-8d45-4d1b-b6ed-31b51c9e13e8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:17.375934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1231/2460] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:03:17.380057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c9881-131c-4d72-a597-ac44acc24c61 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:17.391647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c9881-131c-4d72-a597-ac44acc24c61 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:17.399642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343bd8f-ff6f-4c5a-901f-87073e3b642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:17.400544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343bd8f-ff6f-4c5a-901f-87073e3b642d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:17.400995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343bd8f-ff6f-4c5a-901f-87073e3b642d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:17.401442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343bd8f-ff6f-4c5a-901f-87073e3b642d admin admin] PATH_INFO: `/v3/projects/a1b74e0af6924eaca6c3e94efb720a76/users/3b52445a83c44ed8ade18cfe23cdac14/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:17.404169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343bd8f-ff6f-4c5a-901f-87073e3b642d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b74e0af6924eaca6c3e94efb720a76, user_id=3b52445a83c44ed8ade18cfe23cdac14, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:17.407126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343bd8f-ff6f-4c5a-901f-87073e3b642d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:17.419018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1230/2461] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:03:17 2022] PUT /identity/v3/projects/a1b74e0af6924eaca6c3e94efb720a76/users/3b52445a83c44ed8ade18cfe23cdac14/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:03:21.079397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bbbe0-8d45-4d1b-b6ed-31b51c9e13e8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:21.082941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-228bbbe0-8d45-4d1b-b6ed-31b51c9e13e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:21.244684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bbbe0-8d45-4d1b-b6ed-31b51c9e13e8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:21.371904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc2732-f69f-4594-bfc7-2877669fd783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:21.372719 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc2732-f69f-4594-bfc7-2877669fd783 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:21.373012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc2732-f69f-4594-bfc7-2877669fd783 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:21.373293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc2732-f69f-4594-bfc7-2877669fd783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:21.375658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f6dc2732-f69f-4594-bfc7-2877669fd783 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:21.459157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:21.528305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc2732-f69f-4594-bfc7-2877669fd783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:21.529785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc2732-f69f-4594-bfc7-2877669fd783 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:21.572850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1232/2462] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:23.690346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343bd8f-ff6f-4c5a-901f-87073e3b642d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:23.703029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343bd8f-ff6f-4c5a-901f-87073e3b642d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:23.707910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:23.708403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:23.708687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:23.708960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:23.711036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:23.781611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:23.783016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:23.808415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1231/2463] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:24.232279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d432ac-dc95-4b2c-847d-c2f210ca831d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:24.232755 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d432ac-dc95-4b2c-847d-c2f210ca831d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:24.233226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d432ac-dc95-4b2c-847d-c2f210ca831d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:24.290585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-73d432ac-dc95-4b2c-847d-c2f210ca831d None None] MFA Rules not processed for user `3b52445a83c44ed8ade18cfe23cdac14`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:24.291995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-73d432ac-dc95-4b2c-847d-c2f210ca831d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:24.398499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1233/2464] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:03:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:24.411406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:24.412873 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:24.476764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6043411-c943-4ad5-bbcb-2b9bed39f438 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:24.480654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e19fc3-3ff3-43e4-a33e-5afa898d566d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '548acf58bf9140ec9f1ba126e10deb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:24.481294 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e19fc3-3ff3-43e4-a33e-5afa898d566d service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:24.481579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e19fc3-3ff3-43e4-a33e-5afa898d566d service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:24.481857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e19fc3-3ff3-43e4-a33e-5afa898d566d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:24.556852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e19fc3-3ff3-43e4-a33e-5afa898d566d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:24.558430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e19fc3-3ff3-43e4-a33e-5afa898d566d service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:24.569200 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1232/2465] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:24.708382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646104d-e789-464b-8bd7-6f6af091ad62 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:24.708911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646104d-e789-464b-8bd7-6f6af091ad62 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:24.709491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646104d-e789-464b-8bd7-6f6af091ad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:24.732561 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a646104d-e789-464b-8bd7-6f6af091ad62 None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:24.733785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a646104d-e789-464b-8bd7-6f6af091ad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:24.817977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1234/2466] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 05:03:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:24.899667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e19fc3-3ff3-43e4-a33e-5afa898d566d service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:24.901296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-15e19fc3-3ff3-43e4-a33e-5afa898d566d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:24.951601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:24.983670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e19fc3-3ff3-43e4-a33e-5afa898d566d service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:24.989842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1275b-e035-4867-a006-ad80a4d99eab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:24.990636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1275b-e035-4867-a006-ad80a4d99eab service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:24.990925 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1275b-e035-4867-a006-ad80a4d99eab service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:24.991233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1275b-e035-4867-a006-ad80a4d99eab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:25.000441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1275b-e035-4867-a006-ad80a4d99eab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:25.002234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff1275b-e035-4867-a006-ad80a4d99eab service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:25.011659 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1233/2467] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:03:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 30 05:03:26.657091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646104d-e789-464b-8bd7-6f6af091ad62 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:26.774398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646104d-e789-464b-8bd7-6f6af091ad62 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:26.780220 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed7c6e-db61-4984-8c8a-62e1e7597e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:26.781454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed7c6e-db61-4984-8c8a-62e1e7597e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:26.782358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed7c6e-db61-4984-8c8a-62e1e7597e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:26.782783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed7c6e-db61-4984-8c8a-62e1e7597e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:26.784975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8bed7c6e-db61-4984-8c8a-62e1e7597e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:26.800370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed7c6e-db61-4984-8c8a-62e1e7597e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:26.802061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed7c6e-db61-4984-8c8a-62e1e7597e49 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:26.835655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1235/2468] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:27.062122 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7e238-3d3d-4986-834c-eea8c3be218e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:27.063177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7e238-3d3d-4986-834c-eea8c3be218e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:27.063952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7e238-3d3d-4986-834c-eea8c3be218e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:27.065822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1234/2469] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 05:03:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 05:03:27.079647 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e537a-cf99-4cf8-9e2a-2d92b68a747d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:27.079965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e537a-cf99-4cf8-9e2a-2d92b68a747d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:27.080258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e537a-cf99-4cf8-9e2a-2d92b68a747d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:27.102862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:27.117396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4e9e537a-cf99-4cf8-9e2a-2d92b68a747d None None] MFA Rules not processed for user `426be56424504e958402d785a068d2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:27.123825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4e9e537a-cf99-4cf8-9e2a-2d92b68a747d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:27.135459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1236/2470] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 05:03:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:27.150639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7e238-3d3d-4986-834c-eea8c3be218e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:27.152416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30f7e238-3d3d-4986-834c-eea8c3be218e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:27.169557 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7e238-3d3d-4986-834c-eea8c3be218e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:27.174621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fa4b2-34f4-4022-ae52-206870a8a2ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:27.175242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fa4b2-34f4-4022-ae52-206870a8a2ae service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:27.175528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fa4b2-34f4-4022-ae52-206870a8a2ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:27.175796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fa4b2-34f4-4022-ae52-206870a8a2ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:27.184621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fa4b2-34f4-4022-ae52-206870a8a2ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:27.186051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0fa4b2-34f4-4022-ae52-206870a8a2ae service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:27.196843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1235/2471] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:27.304762 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970da852-5e65-447e-8d08-6d4ac92f6512 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:27.305040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970da852-5e65-447e-8d08-6d4ac92f6512 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:27.305318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970da852-5e65-447e-8d08-6d4ac92f6512 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:27.307332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1237/2472] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 05:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 05:03:27.315445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7121e14-598b-46c0-a2b5-6e36d96f48dc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:27.315731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7121e14-598b-46c0-a2b5-6e36d96f48dc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:27.317522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7121e14-598b-46c0-a2b5-6e36d96f48dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:27.338523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e7121e14-598b-46c0-a2b5-6e36d96f48dc None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:27.340063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7121e14-598b-46c0-a2b5-6e36d96f48dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:27.432619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1236/2473] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 05:03:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:27.443132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970da852-5e65-447e-8d08-6d4ac92f6512 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:27.455731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970da852-5e65-447e-8d08-6d4ac92f6512 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:27.459662 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4e3d7-ebbf-4427-a2f1-c23697763dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:27.460888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4e3d7-ebbf-4427-a2f1-c23697763dbf service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:27.460888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4e3d7-ebbf-4427-a2f1-c23697763dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:27.461611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4e3d7-ebbf-4427-a2f1-c23697763dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:27.473061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4e3d7-ebbf-4427-a2f1-c23697763dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:27.474537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4e3d7-ebbf-4427-a2f1-c23697763dbf service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:27.486738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1238/2474] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:27.558713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7121e14-598b-46c0-a2b5-6e36d96f48dc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:27.560202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7121e14-598b-46c0-a2b5-6e36d96f48dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:27.643878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7121e14-598b-46c0-a2b5-6e36d96f48dc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:27.649491 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191d6e5-3d87-492b-b088-e8ccd76beb2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:27.650193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191d6e5-3d87-492b-b088-e8ccd76beb2a service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:27.651205 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191d6e5-3d87-492b-b088-e8ccd76beb2a service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:27.651562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191d6e5-3d87-492b-b088-e8ccd76beb2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:27.653463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8191d6e5-3d87-492b-b088-e8ccd76beb2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:27.669974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191d6e5-3d87-492b-b088-e8ccd76beb2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:27.672177 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191d6e5-3d87-492b-b088-e8ccd76beb2a service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:27.684689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1237/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:36.008677 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0aef2-92cf-4402-848d-09c70631698b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:36.009599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0aef2-92cf-4402-848d-09c70631698b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:36.010028 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0aef2-92cf-4402-848d-09c70631698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:36.031569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-95e0aef2-92cf-4402-848d-09c70631698b None None] MFA Rules not processed for user `42d3708ddfcc491cb366054dcd5f8881`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:36.033801 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-95e0aef2-92cf-4402-848d-09c70631698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:36.041217 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1239/2476] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:03:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:37.680416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:40.183439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c8d4e-87d1-4eb4-9800-ff06aac4843a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:40.184041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c8d4e-87d1-4eb4-9800-ff06aac4843a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:40.184041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c8d4e-87d1-4eb4-9800-ff06aac4843a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:40.205406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b30c8d4e-87d1-4eb4-9800-ff06aac4843a None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:40.206764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b30c8d4e-87d1-4eb4-9800-ff06aac4843a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:40.214099 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1238/2477] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 05:03:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:40.226691 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0aef2-92cf-4402-848d-09c70631698b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:40.239745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0aef2-92cf-4402-848d-09c70631698b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:40.245022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7e85f-6deb-4b68-9fc9-96c2d0687aab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:40.245644 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7e85f-6deb-4b68-9fc9-96c2d0687aab service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:40.245918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7e85f-6deb-4b68-9fc9-96c2d0687aab service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:40.246185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7e85f-6deb-4b68-9fc9-96c2d0687aab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:40.256335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7e85f-6deb-4b68-9fc9-96c2d0687aab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:40.257965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7e85f-6deb-4b68-9fc9-96c2d0687aab service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:40.266370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1240/2478] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:03:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 30 05:03:43.598378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1d173-2a82-4fd7-923f-a8f9bb1682b7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:43.599800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1d173-2a82-4fd7-923f-a8f9bb1682b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:43.599800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1d173-2a82-4fd7-923f-a8f9bb1682b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:43.627118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-05f1d173-2a82-4fd7-923f-a8f9bb1682b7 None None] MFA Rules not processed for user `a8abb58f9c274e568d2d0fe3d4df11de`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:03:43.628988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-05f1d173-2a82-4fd7-923f-a8f9bb1682b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:43.637288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1239/2479] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:03:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:03:49.316688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7e85f-6deb-4b68-9fc9-96c2d0687aab service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:49.326951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7e85f-6deb-4b68-9fc9-96c2d0687aab service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:49.331302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:49.331913 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:49.332192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:49.332463 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:49.334251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:49.344041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:49.345457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:49.353998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1241/2480] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:49.922301 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1d173-2a82-4fd7-923f-a8f9bb1682b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:49.925191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-05f1d173-2a82-4fd7-923f-a8f9bb1682b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:49.946965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1d173-2a82-4fd7-923f-a8f9bb1682b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:49.954063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faf9b3-a6a8-4fa2-b48a-176fb6dcfe2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:49.955044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf9b3-a6a8-4fa2-b48a-176fb6dcfe2e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:49.955536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf9b3-a6a8-4fa2-b48a-176fb6dcfe2e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:49.955987 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faf9b3-a6a8-4fa2-b48a-176fb6dcfe2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:49.958023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-13faf9b3-a6a8-4fa2-b48a-176fb6dcfe2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:49.974321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faf9b3-a6a8-4fa2-b48a-176fb6dcfe2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:49.976022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13faf9b3-a6a8-4fa2-b48a-176fb6dcfe2e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:49.989197 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1240/2481] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:03:57.505445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:57.506941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:57.518570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=23 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:03:57.526098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daa87f-2bc5-4532-ba45-11014a6462d8 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:57.532830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4583081-9773-4fc3-b6bd-292d01e04bba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:57.534082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4583081-9773-4fc3-b6bd-292d01e04bba service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:57.534326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4583081-9773-4fc3-b6bd-292d01e04bba service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:57.534669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4583081-9773-4fc3-b6bd-292d01e04bba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:57.536709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e4583081-9773-4fc3-b6bd-292d01e04bba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:57.614576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4583081-9773-4fc3-b6bd-292d01e04bba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:57.616324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4583081-9773-4fc3-b6bd-292d01e04bba service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:57.630477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1242/2482] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:03:59.455765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faf9b3-a6a8-4fa2-b48a-176fb6dcfe2e service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:03:59.465351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13faf9b3-a6a8-4fa2-b48a-176fb6dcfe2e service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:03:59.469007 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:03:59.469612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:03:59.469888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:03:59.470675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:03:59.472690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:03:59.544367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:03:59.546787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:03:59.557583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1241/2483] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:03:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:03.101109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4583081-9773-4fc3-b6bd-292d01e04bba service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:03.117693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4583081-9773-4fc3-b6bd-292d01e04bba service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:03.123532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8773f4-e76a-43ab-990d-07ea3f10ff5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:03.124176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8773f4-e76a-43ab-990d-07ea3f10ff5b service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:03.124754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8773f4-e76a-43ab-990d-07ea3f10ff5b service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:03.125501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8773f4-e76a-43ab-990d-07ea3f10ff5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:03.128114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6d8773f4-e76a-43ab-990d-07ea3f10ff5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:03.144001 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8773f4-e76a-43ab-990d-07ea3f10ff5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:03.145605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8773f4-e76a-43ab-990d-07ea3f10ff5b service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:03.160469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1243/2484] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:03.175507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:03.177011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:03.198938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ba2a4-2921-495d-a993-066c9086d141 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:03.206551 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:03.207212 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:03.207488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:03.207758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] PATH_INFO: `/v3/users/25db6f7402ae4a14a04fab0a9915ab77` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:03.208937 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25db6f7402ae4a14a04fab0a9915ab77)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:03.210932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:03.263479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25db6f7402ae4a14a04fab0a9915ab77'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.271288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25db6f7402ae4a14a04fab0a9915ab77'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.272039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25db6f7402ae4a14a04fab0a9915ab77'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.272717 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25db6f7402ae4a14a04fab0a9915ab77'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.287316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ed640c9-51d3-400c-a9aa-5f76f2e189f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25db6f7402ae4a14a04fab0a9915ab77'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.302362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1242/2485] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:03 2022] DELETE /identity/v3/users/25db6f7402ae4a14a04fab0a9915ab77 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:03.556469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8773f4-e76a-43ab-990d-07ea3f10ff5b service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:03.567277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8773f4-e76a-43ab-990d-07ea3f10ff5b service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:03.638940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:03.639576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:03.639854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:03.640124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] PATH_INFO: `/v3/projects/decb0bd64582440590890900e594b79e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:03.641268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=decb0bd64582440590890900e594b79e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:03.643878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:03.692056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'decb0bd64582440590890900e594b79e', 'request_id': 'req-e4347cef-b555-4acb-be2a-ac6ae68bf847'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.701933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'decb0bd64582440590890900e594b79e', 'request_id': 'req-e4347cef-b555-4acb-be2a-ac6ae68bf847'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.711500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'decb0bd64582440590890900e594b79e', 'request_id': 'req-e4347cef-b555-4acb-be2a-ac6ae68bf847'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.724439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] The token cache is being invalidate because project decb0bd64582440590890900e594b79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:03.724724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.725371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4347cef-b555-4acb-be2a-ac6ae68bf847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:03.726941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1244/2486] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:03 2022] DELETE /identity/v3/projects/decb0bd64582440590890900e594b79e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:03.731830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f662d8-3536-4801-82e3-cac7c0216cc9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:03.732147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f662d8-3536-4801-82e3-cac7c0216cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:03.732425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f662d8-3536-4801-82e3-cac7c0216cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:03.753912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f5f662d8-3536-4801-82e3-cac7c0216cc9 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:03.755287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5f662d8-3536-4801-82e3-cac7c0216cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:03.791071 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:03.820436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1243/2487] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:03.828197 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af0d9a-eaeb-4af2-a0a6-3f3cf186da00 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:03.828478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af0d9a-eaeb-4af2-a0a6-3f3cf186da00 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:03.828752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af0d9a-eaeb-4af2-a0a6-3f3cf186da00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:03.851942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b0af0d9a-eaeb-4af2-a0a6-3f3cf186da00 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:03.853296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b0af0d9a-eaeb-4af2-a0a6-3f3cf186da00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:03.863052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1245/2488] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:03.867947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f662d8-3536-4801-82e3-cac7c0216cc9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:03.878231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f662d8-3536-4801-82e3-cac7c0216cc9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:03.882983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7a015-c4c7-4bb3-9718-b2b2db41dff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:03.883593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7a015-c4c7-4bb3-9718-b2b2db41dff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:03.883870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7a015-c4c7-4bb3-9718-b2b2db41dff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:03.884141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7a015-c4c7-4bb3-9718-b2b2db41dff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:03.884862 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7a015-c4c7-4bb3-9718-b2b2db41dff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:04:03.885163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7a015-c4c7-4bb3-9718-b2b2db41dff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:03.886796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7a015-c4c7-4bb3-9718-b2b2db41dff8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:03.898914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1244/2489] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 05:04:03.903384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af0d9a-eaeb-4af2-a0a6-3f3cf186da00 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:03.912962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af0d9a-eaeb-4af2-a0a6-3f3cf186da00 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:03.917762 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a1d6e-ac79-40fb-b3de-7f8987bf96b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:03.918364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a1d6e-ac79-40fb-b3de-7f8987bf96b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:03.918683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a1d6e-ac79-40fb-b3de-7f8987bf96b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:03.918953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a1d6e-ac79-40fb-b3de-7f8987bf96b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:03.919607 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a1d6e-ac79-40fb-b3de-7f8987bf96b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:03.921764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a1d6e-ac79-40fb-b3de-7f8987bf96b3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:03.931604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:03.950528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1246/2490] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:03 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:03.955637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7a015-c4c7-4bb3-9718-b2b2db41dff8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:03.965803 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7a015-c4c7-4bb3-9718-b2b2db41dff8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:04.022167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ec16f-66cd-48d0-9d7e-018a25db68a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:04.022822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ec16f-66cd-48d0-9d7e-018a25db68a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:04.023102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ec16f-66cd-48d0-9d7e-018a25db68a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:04.023372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ec16f-66cd-48d0-9d7e-018a25db68a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:04.024042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ec16f-66cd-48d0-9d7e-018a25db68a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:04.026011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ec16f-66cd-48d0-9d7e-018a25db68a8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:04.047376 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1245/2491] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:03 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:04.051489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a1d6e-ac79-40fb-b3de-7f8987bf96b3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:04.061740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a1d6e-ac79-40fb-b3de-7f8987bf96b3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:04.066936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106d0d6-3bc2-4ea6-9b35-5918b9bc37cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:04.067549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d0d6-3bc2-4ea6-9b35-5918b9bc37cb admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:04.067826 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d0d6-3bc2-4ea6-9b35-5918b9bc37cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:04.068096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d0d6-3bc2-4ea6-9b35-5918b9bc37cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:04.068787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106d0d6-3bc2-4ea6-9b35-5918b9bc37cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:04.070407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106d0d6-3bc2-4ea6-9b35-5918b9bc37cb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:04.081066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1247/2492] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:04.084709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ec16f-66cd-48d0-9d7e-018a25db68a8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:04.094753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ec16f-66cd-48d0-9d7e-018a25db68a8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:04.099675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2cc66-9c23-4945-87b4-2053a340383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:04.100276 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2cc66-9c23-4945-87b4-2053a340383a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:04.100555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2cc66-9c23-4945-87b4-2053a340383a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:04.100828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2cc66-9c23-4945-87b4-2053a340383a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:04.101519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2cc66-9c23-4945-87b4-2053a340383a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:04.103156 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f2cc66-9c23-4945-87b4-2053a340383a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:04.115100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1246/2493] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:04.119200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106d0d6-3bc2-4ea6-9b35-5918b9bc37cb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:04.129797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106d0d6-3bc2-4ea6-9b35-5918b9bc37cb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:04.134473 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175706c-a117-4de0-9cc4-bf34da2b49cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:04.135122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175706c-a117-4de0-9cc4-bf34da2b49cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:04.135400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175706c-a117-4de0-9cc4-bf34da2b49cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:04.135671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175706c-a117-4de0-9cc4-bf34da2b49cc admin admin] PATH_INFO: `/v3/projects/dc7891287dd64eadac8de9cdf8e41b74/users/3b26db3178574782bee6b7f8fbc15e64/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:04.150969 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175706c-a117-4de0-9cc4-bf34da2b49cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7891287dd64eadac8de9cdf8e41b74, user_id=3b26db3178574782bee6b7f8fbc15e64, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:04.153615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175706c-a117-4de0-9cc4-bf34da2b49cc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:04.166506 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1248/2494] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:04 2022] PUT /identity/v3/projects/dc7891287dd64eadac8de9cdf8e41b74/users/3b26db3178574782bee6b7f8fbc15e64/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:04.178035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55310fde-c55e-432a-9ef5-76664508fb91 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:04.178562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55310fde-c55e-432a-9ef5-76664508fb91 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:04.178662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55310fde-c55e-432a-9ef5-76664508fb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:04.206807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-55310fde-c55e-432a-9ef5-76664508fb91 None None] MFA Rules not processed for user `3b26db3178574782bee6b7f8fbc15e64`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:04.208022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55310fde-c55e-432a-9ef5-76664508fb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:04.272646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1247/2495] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:04.284123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175706c-a117-4de0-9cc4-bf34da2b49cc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:04.285835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1175706c-a117-4de0-9cc4-bf34da2b49cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:04.359133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175706c-a117-4de0-9cc4-bf34da2b49cc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:04.365670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f179-40f8-4ba4-b749-93d193451915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:04.366278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f179-40f8-4ba4-b749-93d193451915 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:04.366583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f179-40f8-4ba4-b749-93d193451915 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:04.366854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f179-40f8-4ba4-b749-93d193451915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:04.381049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761f179-40f8-4ba4-b749-93d193451915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:04.383021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761f179-40f8-4ba4-b749-93d193451915 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:04.391915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1249/2496] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:07.523636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55310fde-c55e-432a-9ef5-76664508fb91 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:07.525181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55310fde-c55e-432a-9ef5-76664508fb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:07.541817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55310fde-c55e-432a-9ef5-76664508fb91 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:07.547697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939ccb0-5a9a-443d-a21f-54b86948e9a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:07.548985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939ccb0-5a9a-443d-a21f-54b86948e9a1 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:07.549328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939ccb0-5a9a-443d-a21f-54b86948e9a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:07.549670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939ccb0-5a9a-443d-a21f-54b86948e9a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:07.551540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3939ccb0-5a9a-443d-a21f-54b86948e9a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:07.564051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939ccb0-5a9a-443d-a21f-54b86948e9a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:07.565709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939ccb0-5a9a-443d-a21f-54b86948e9a1 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:07.576252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1248/2497] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:10.371091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f179-40f8-4ba4-b749-93d193451915 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:10.372516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8761f179-40f8-4ba4-b749-93d193451915 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:10.430754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f179-40f8-4ba4-b749-93d193451915 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:10.434714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08016c3e-dd42-4b35-a3af-0f3d8bcd9ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:10.435350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016c3e-dd42-4b35-a3af-0f3d8bcd9ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:10.435630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016c3e-dd42-4b35-a3af-0f3d8bcd9ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:10.435946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016c3e-dd42-4b35-a3af-0f3d8bcd9ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:10.437738 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-08016c3e-dd42-4b35-a3af-0f3d8bcd9ec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:10.498397 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:10.514287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08016c3e-dd42-4b35-a3af-0f3d8bcd9ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:10.515841 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08016c3e-dd42-4b35-a3af-0f3d8bcd9ec5 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:10.528552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1250/2498] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:11.748432 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939ccb0-5a9a-443d-a21f-54b86948e9a1 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:11.774667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:11.814903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939ccb0-5a9a-443d-a21f-54b86948e9a1 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:11.819591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:11.820259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:11.820541 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:11.820817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] PATH_INFO: `/v3/users/3b26db3178574782bee6b7f8fbc15e64` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:11.822118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b26db3178574782bee6b7f8fbc15e64)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:11.824083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:11.875154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b26db3178574782bee6b7f8fbc15e64'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:11.883238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b26db3178574782bee6b7f8fbc15e64'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:11.883968 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b26db3178574782bee6b7f8fbc15e64'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:11.884597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b26db3178574782bee6b7f8fbc15e64'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:11.899953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b26db3178574782bee6b7f8fbc15e64'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:11.918010 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1249/2499] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:11 2022] DELETE /identity/v3/users/3b26db3178574782bee6b7f8fbc15e64 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:11.923017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c29e-109e-4712-856f-9c62bc67278f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:11.923505 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c29e-109e-4712-856f-9c62bc67278f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:11.923895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c29e-109e-4712-856f-9c62bc67278f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:11.943591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4992c29e-109e-4712-856f-9c62bc67278f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:11.944906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4992c29e-109e-4712-856f-9c62bc67278f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:12.011605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1251/2500] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:12.022232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:12.023813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:12.084471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596194eb-da7c-4239-9956-d3660a2955b1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:12.088199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101e16c-3a36-4219-8901-43eaa6a9caf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:12.088826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101e16c-3a36-4219-8901-43eaa6a9caf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:12.089109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101e16c-3a36-4219-8901-43eaa6a9caf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:12.089388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101e16c-3a36-4219-8901-43eaa6a9caf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:12.098627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101e16c-3a36-4219-8901-43eaa6a9caf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:12.100068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101e16c-3a36-4219-8901-43eaa6a9caf5 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:12.110532 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1250/2501] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:12.460745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c29e-109e-4712-856f-9c62bc67278f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:12.462454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4992c29e-109e-4712-856f-9c62bc67278f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:12.489924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4992c29e-109e-4712-856f-9c62bc67278f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:12.495936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:12.496669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:12.497016 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:12.497398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] PATH_INFO: `/v3/projects/dc7891287dd64eadac8de9cdf8e41b74` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:12.498919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7891287dd64eadac8de9cdf8e41b74)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:12.502118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:12.595688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7891287dd64eadac8de9cdf8e41b74', 'request_id': 'req-44fe2f35-499d-45df-bccf-f156b930ce12'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:12.606196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7891287dd64eadac8de9cdf8e41b74', 'request_id': 'req-44fe2f35-499d-45df-bccf-f156b930ce12'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:12.614134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7891287dd64eadac8de9cdf8e41b74', 'request_id': 'req-44fe2f35-499d-45df-bccf-f156b930ce12'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:12.628444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] The token cache is being invalidate because project dc7891287dd64eadac8de9cdf8e41b74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:12.628881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:12.629737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-44fe2f35-499d-45df-bccf-f156b930ce12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:12.635859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1252/2502] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:12 2022] DELETE /identity/v3/projects/dc7891287dd64eadac8de9cdf8e41b74 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:12.642765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f3181-8716-4f03-a5e1-1ceda75dcc6e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:12.642765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f3181-8716-4f03-a5e1-1ceda75dcc6e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:12.643977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f3181-8716-4f03-a5e1-1ceda75dcc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:12.687109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6a4f3181-8716-4f03-a5e1-1ceda75dcc6e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:12.696208 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6a4f3181-8716-4f03-a5e1-1ceda75dcc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:12.752777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:12.780768 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1251/2503] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:12.789975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc098b6-c760-4f49-9168-41b74c243960 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:12.790316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc098b6-c760-4f49-9168-41b74c243960 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:12.790569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc098b6-c760-4f49-9168-41b74c243960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:12.834087 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5bc098b6-c760-4f49-9168-41b74c243960 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:12.836364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5bc098b6-c760-4f49-9168-41b74c243960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:12.849366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1253/2504] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:12.851016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f3181-8716-4f03-a5e1-1ceda75dcc6e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:12.864509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f3181-8716-4f03-a5e1-1ceda75dcc6e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:12.871391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88aa0b-0467-494d-ad0c-927956aea4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:12.872706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88aa0b-0467-494d-ad0c-927956aea4af admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:12.873003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88aa0b-0467-494d-ad0c-927956aea4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:12.873508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88aa0b-0467-494d-ad0c-927956aea4af admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:12.874749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88aa0b-0467-494d-ad0c-927956aea4af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:04:12.875471 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88aa0b-0467-494d-ad0c-927956aea4af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:12.877345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88aa0b-0467-494d-ad0c-927956aea4af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:12.890530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1252/2505] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 05:04:12.894637 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc098b6-c760-4f49-9168-41b74c243960 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:12.906384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc098b6-c760-4f49-9168-41b74c243960 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:12.914056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536046d5-2554-41b2-8df7-eb7201122222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:12.914993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536046d5-2554-41b2-8df7-eb7201122222 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:12.915476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536046d5-2554-41b2-8df7-eb7201122222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:12.916036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536046d5-2554-41b2-8df7-eb7201122222 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:12.916906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536046d5-2554-41b2-8df7-eb7201122222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:12.919487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536046d5-2554-41b2-8df7-eb7201122222 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:12.954323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1254/2506] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:12 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:12.958913 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88aa0b-0467-494d-ad0c-927956aea4af admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:12.970665 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88aa0b-0467-494d-ad0c-927956aea4af admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:13.065516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c76ac5-c549-4919-97b6-1e59ff14b64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:13.066129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c76ac5-c549-4919-97b6-1e59ff14b64e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.066733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c76ac5-c549-4919-97b6-1e59ff14b64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.067049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c76ac5-c549-4919-97b6-1e59ff14b64e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.067734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c76ac5-c549-4919-97b6-1e59ff14b64e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:13.070353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c76ac5-c549-4919-97b6-1e59ff14b64e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:13.106346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1253/2507] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:12 2022] POST /identity/v3/users => generated 284 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:13.110997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536046d5-2554-41b2-8df7-eb7201122222 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:13.124549 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536046d5-2554-41b2-8df7-eb7201122222 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:13.130033 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15089a0-82f4-4e0f-8e00-430cf0a6e95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:13.130731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15089a0-82f4-4e0f-8e00-430cf0a6e95f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.131012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15089a0-82f4-4e0f-8e00-430cf0a6e95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.131286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15089a0-82f4-4e0f-8e00-430cf0a6e95f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.131990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15089a0-82f4-4e0f-8e00-430cf0a6e95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:13.133643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15089a0-82f4-4e0f-8e00-430cf0a6e95f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:13.151593 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1255/2508] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:13.155279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c76ac5-c549-4919-97b6-1e59ff14b64e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:13.171341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c76ac5-c549-4919-97b6-1e59ff14b64e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:13.177570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19930f5c-5dec-45f3-aee1-3445ee91df19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:13.178306 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19930f5c-5dec-45f3-aee1-3445ee91df19 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.178646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19930f5c-5dec-45f3-aee1-3445ee91df19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.178980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19930f5c-5dec-45f3-aee1-3445ee91df19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.179928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19930f5c-5dec-45f3-aee1-3445ee91df19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:13.183659 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19930f5c-5dec-45f3-aee1-3445ee91df19 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:13.202101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1254/2509] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:13.207256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15089a0-82f4-4e0f-8e00-430cf0a6e95f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:13.226095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15089a0-82f4-4e0f-8e00-430cf0a6e95f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:13.232556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c42392-2bbd-4299-abd5-e2fc8b42693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:13.233546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c42392-2bbd-4299-abd5-e2fc8b42693f admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.233859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c42392-2bbd-4299-abd5-e2fc8b42693f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.234276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c42392-2bbd-4299-abd5-e2fc8b42693f admin admin] PATH_INFO: `/v3/projects/09a7beecab60456ab0cfb9865d7c1b99/users/49769fd54b07492ab36d9ea3fdd345d0/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.267761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c42392-2bbd-4299-abd5-e2fc8b42693f admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a7beecab60456ab0cfb9865d7c1b99, user_id=49769fd54b07492ab36d9ea3fdd345d0, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:13.271686 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c42392-2bbd-4299-abd5-e2fc8b42693f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:13.298458 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1256/2510] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:13 2022] PUT /identity/v3/projects/09a7beecab60456ab0cfb9865d7c1b99/users/49769fd54b07492ab36d9ea3fdd345d0/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:13.309138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661b00f-a44e-42d1-9f9d-207afea75eaf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.309504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661b00f-a44e-42d1-9f9d-207afea75eaf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.309710 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661b00f-a44e-42d1-9f9d-207afea75eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.354773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2661b00f-a44e-42d1-9f9d-207afea75eaf None None] MFA Rules not processed for user `49769fd54b07492ab36d9ea3fdd345d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:13.356241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2661b00f-a44e-42d1-9f9d-207afea75eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:13.431416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1255/2511] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:13.436007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c42392-2bbd-4299-abd5-e2fc8b42693f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:13.448104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:13.525492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c42392-2bbd-4299-abd5-e2fc8b42693f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:13.534820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019e6b4-5f57-4942-8b73-2752b943b12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:13.535554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019e6b4-5f57-4942-8b73-2752b943b12b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.535974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019e6b4-5f57-4942-8b73-2752b943b12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.536152 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019e6b4-5f57-4942-8b73-2752b943b12b admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.536786 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019e6b4-5f57-4942-8b73-2752b943b12b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:13.538735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019e6b4-5f57-4942-8b73-2752b943b12b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:13.574500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1257/2512] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:13 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:13.575758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661b00f-a44e-42d1-9f9d-207afea75eaf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:13.591260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661b00f-a44e-42d1-9f9d-207afea75eaf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:13.640687 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:13.666780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc3475-10b5-4941-bc31-fd0cbd009ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:13.667424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc3475-10b5-4941-bc31-fd0cbd009ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.667708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc3475-10b5-4941-bc31-fd0cbd009ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.667988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc3475-10b5-4941-bc31-fd0cbd009ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.668746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc3475-10b5-4941-bc31-fd0cbd009ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:13.671674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc3475-10b5-4941-bc31-fd0cbd009ff0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:13.699755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1256/2513] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:13 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:13.701215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019e6b4-5f57-4942-8b73-2752b943b12b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:13.712324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019e6b4-5f57-4942-8b73-2752b943b12b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:13.717335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe622a-27c6-428a-860c-c2a506dd3c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:13.717962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe622a-27c6-428a-860c-c2a506dd3c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.718240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe622a-27c6-428a-860c-c2a506dd3c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.718550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe622a-27c6-428a-860c-c2a506dd3c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.719252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe622a-27c6-428a-860c-c2a506dd3c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:13.721791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe622a-27c6-428a-860c-c2a506dd3c7e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:13.737955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1258/2514] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:04:13.742176 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc3475-10b5-4941-bc31-fd0cbd009ff0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:13.752938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc3475-10b5-4941-bc31-fd0cbd009ff0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:13.757632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82bf0d-6a6e-4cb8-967e-041a9b916dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:13.758292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82bf0d-6a6e-4cb8-967e-041a9b916dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.758631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82bf0d-6a6e-4cb8-967e-041a9b916dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.758909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82bf0d-6a6e-4cb8-967e-041a9b916dc0 admin admin] PATH_INFO: `/v3/projects/c824b16f67f84b4195cf8c7c981749ac/users/c20863b671da49ca8463b4139f2c4673/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.905314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82bf0d-6a6e-4cb8-967e-041a9b916dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c824b16f67f84b4195cf8c7c981749ac, user_id=c20863b671da49ca8463b4139f2c4673, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:13.908636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82bf0d-6a6e-4cb8-967e-041a9b916dc0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:13.922001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1257/2515] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:13 2022] PUT /identity/v3/projects/c824b16f67f84b4195cf8c7c981749ac/users/c20863b671da49ca8463b4139f2c4673/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:04:13.926124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe622a-27c6-428a-860c-c2a506dd3c7e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:13.936302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe622a-27c6-428a-860c-c2a506dd3c7e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:13.993611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3174076-5efc-4eea-87cb-d09e6a8aa042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:13.994276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3174076-5efc-4eea-87cb-d09e6a8aa042 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:13.994520 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3174076-5efc-4eea-87cb-d09e6a8aa042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:13.994798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3174076-5efc-4eea-87cb-d09e6a8aa042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:13.995497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3174076-5efc-4eea-87cb-d09e6a8aa042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:13.997137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3174076-5efc-4eea-87cb-d09e6a8aa042 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:14.013035 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1259/2516] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:04:14.017435 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82bf0d-6a6e-4cb8-967e-041a9b916dc0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:14.029055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82bf0d-6a6e-4cb8-967e-041a9b916dc0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:14.034258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734dcf3-3612-444e-9301-a1464abf87f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:14.035132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734dcf3-3612-444e-9301-a1464abf87f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:14.035548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734dcf3-3612-444e-9301-a1464abf87f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:14.035822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734dcf3-3612-444e-9301-a1464abf87f3 admin admin] PATH_INFO: `/v3/projects/c824b16f67f84b4195cf8c7c981749ac/users/c20863b671da49ca8463b4139f2c4673/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:14.037786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734dcf3-3612-444e-9301-a1464abf87f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c824b16f67f84b4195cf8c7c981749ac, user_id=c20863b671da49ca8463b4139f2c4673, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:14.040543 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734dcf3-3612-444e-9301-a1464abf87f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:14.052331 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1258/2517] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:14 2022] PUT /identity/v3/projects/c824b16f67f84b4195cf8c7c981749ac/users/c20863b671da49ca8463b4139f2c4673/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:14.064522 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2dd3-3d96-452e-957e-8318df4ab0bd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:14.064933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2dd3-3d96-452e-957e-8318df4ab0bd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:14.065259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2dd3-3d96-452e-957e-8318df4ab0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:14.095859 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-013a2dd3-3d96-452e-957e-8318df4ab0bd None None] MFA Rules not processed for user `c20863b671da49ca8463b4139f2c4673`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:14.097019 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-013a2dd3-3d96-452e-957e-8318df4ab0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:14.169939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1260/2518] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:14.184328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734dcf3-3612-444e-9301-a1464abf87f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:14.186412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3734dcf3-3612-444e-9301-a1464abf87f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:14.286530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734dcf3-3612-444e-9301-a1464abf87f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:14.290818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fd1a8-8aea-40c6-885f-ac03b1e75ce6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:14.291762 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fd1a8-8aea-40c6-885f-ac03b1e75ce6 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:14.292309 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fd1a8-8aea-40c6-885f-ac03b1e75ce6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:14.293035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fd1a8-8aea-40c6-885f-ac03b1e75ce6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:14.309076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fd1a8-8aea-40c6-885f-ac03b1e75ce6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:14.311084 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fd1a8-8aea-40c6-885f-ac03b1e75ce6 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:14.327970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1259/2519] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:15.824584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2dd3-3d96-452e-957e-8318df4ab0bd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:15.830689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:15.890844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2dd3-3d96-452e-957e-8318df4ab0bd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:15.896422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:15.897057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:15.897335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:15.897608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] PATH_INFO: `/v3/users/49769fd54b07492ab36d9ea3fdd345d0` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:15.898792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=49769fd54b07492ab36d9ea3fdd345d0)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:15.900700 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:15.945901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49769fd54b07492ab36d9ea3fdd345d0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:15.952726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49769fd54b07492ab36d9ea3fdd345d0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:15.953382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49769fd54b07492ab36d9ea3fdd345d0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:15.953987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49769fd54b07492ab36d9ea3fdd345d0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:15.966331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49769fd54b07492ab36d9ea3fdd345d0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:15.979283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1261/2520] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:15 2022] DELETE /identity/v3/users/49769fd54b07492ab36d9ea3fdd345d0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:15.983904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab3a05-862d-488d-9119-ddba3d5d2af3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:15.984387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab3a05-862d-488d-9119-ddba3d5d2af3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:15.984485 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab3a05-862d-488d-9119-ddba3d5d2af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:16.005191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5bab3a05-862d-488d-9119-ddba3d5d2af3 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:16.006420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bab3a05-862d-488d-9119-ddba3d5d2af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:16.072321 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1260/2521] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:16.083096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:16.084495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:16.141422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4e070-dc9b-4419-8cd6-6ff3a96885cf admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:16.145160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a13b26-cdd5-44c5-a029-535210c3133c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:16.145766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a13b26-cdd5-44c5-a029-535210c3133c service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:16.146040 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a13b26-cdd5-44c5-a029-535210c3133c service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:16.146307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a13b26-cdd5-44c5-a029-535210c3133c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:16.155302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a13b26-cdd5-44c5-a029-535210c3133c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:16.156732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a13b26-cdd5-44c5-a029-535210c3133c service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:16.166405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1262/2522] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:16.436169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab3a05-862d-488d-9119-ddba3d5d2af3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:16.437602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bab3a05-862d-488d-9119-ddba3d5d2af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:16.452677 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab3a05-862d-488d-9119-ddba3d5d2af3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:16.457358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:16.458025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:16.458290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:16.458602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] PATH_INFO: `/v3/projects/09a7beecab60456ab0cfb9865d7c1b99` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:16.459776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a7beecab60456ab0cfb9865d7c1b99)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:16.463116 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:16.496960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:16.512701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a7beecab60456ab0cfb9865d7c1b99', 'request_id': 'req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.521032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a7beecab60456ab0cfb9865d7c1b99', 'request_id': 'req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.527162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a7beecab60456ab0cfb9865d7c1b99', 'request_id': 'req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.538903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] The token cache is being invalidate because project 09a7beecab60456ab0cfb9865d7c1b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:16.539214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.539843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.541333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1261/2523] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:16 2022] DELETE /identity/v3/projects/09a7beecab60456ab0cfb9865d7c1b99 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:16.545500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a13b26-cdd5-44c5-a029-535210c3133c service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:16.556611 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a13b26-cdd5-44c5-a029-535210c3133c service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:16.580074 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:16.616229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:16.616866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:16.617170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:16.617443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] PATH_INFO: `/v3/users/c20863b671da49ca8463b4139f2c4673` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:16.618770 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c20863b671da49ca8463b4139f2c4673)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:16.620737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:16.667119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c20863b671da49ca8463b4139f2c4673'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.674858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c20863b671da49ca8463b4139f2c4673'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.675518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c20863b671da49ca8463b4139f2c4673'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.676120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c20863b671da49ca8463b4139f2c4673'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.687971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-fa281632-2ca1-4dfe-b3f8-71c6737a841d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c20863b671da49ca8463b4139f2c4673'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:16.700823 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1263/2524] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:16 2022] DELETE /identity/v3/users/c20863b671da49ca8463b4139f2c4673 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:16.979505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:16.981072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:17.003819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dab193-2eb5-4a9d-bd45-d4bfbfa87410 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.073501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:17.074147 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.074428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.074726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] PATH_INFO: `/v3/projects/c824b16f67f84b4195cf8c7c981749ac` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.075908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c824b16f67f84b4195cf8c7c981749ac)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:17.077833 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:17.134787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c824b16f67f84b4195cf8c7c981749ac', 'request_id': 'req-5a1893f3-5261-44cd-bdf0-69f05d3011cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:17.145133 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c824b16f67f84b4195cf8c7c981749ac', 'request_id': 'req-5a1893f3-5261-44cd-bdf0-69f05d3011cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:17.152210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c824b16f67f84b4195cf8c7c981749ac', 'request_id': 'req-5a1893f3-5261-44cd-bdf0-69f05d3011cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:17.165135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] The token cache is being invalidate because project c824b16f67f84b4195cf8c7c981749ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:17.165718 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:17.166519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a1893f3-5261-44cd-bdf0-69f05d3011cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:17.168752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1262/2525] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:16 2022] DELETE /identity/v3/projects/c824b16f67f84b4195cf8c7c981749ac => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:17.174537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa4bda-236c-4490-a8cb-6fc4491c3f0f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.175529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa4bda-236c-4490-a8cb-6fc4491c3f0f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.175834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa4bda-236c-4490-a8cb-6fc4491c3f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.203009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-defa4bda-236c-4490-a8cb-6fc4491c3f0f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:17.204428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-defa4bda-236c-4490-a8cb-6fc4491c3f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:17.274150 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1264/2526] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:17.282562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e9a0d-bd0f-4a55-860e-a51c79534e97 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.283628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e9a0d-bd0f-4a55-860e-a51c79534e97 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.284005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e9a0d-bd0f-4a55-860e-a51c79534e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.305145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f83e9a0d-bd0f-4a55-860e-a51c79534e97 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:17.306359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f83e9a0d-bd0f-4a55-860e-a51c79534e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:17.315845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1263/2527] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:17.319937 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa4bda-236c-4490-a8cb-6fc4491c3f0f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:17.333761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa4bda-236c-4490-a8cb-6fc4491c3f0f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.338979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6e35e-e9ac-45fa-9b4d-ba4aefae7997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:17.339596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e35e-e9ac-45fa-9b4d-ba4aefae7997 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.339874 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e35e-e9ac-45fa-9b4d-ba4aefae7997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.340148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6e35e-e9ac-45fa-9b4d-ba4aefae7997 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.340901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6e35e-e9ac-45fa-9b4d-ba4aefae7997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:04:17.341207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6e35e-e9ac-45fa-9b4d-ba4aefae7997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:17.343007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6e35e-e9ac-45fa-9b4d-ba4aefae7997 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:17.357820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1265/2528] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:04:17.362429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e9a0d-bd0f-4a55-860e-a51c79534e97 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:17.372291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e9a0d-bd0f-4a55-860e-a51c79534e97 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.377667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1acff35-0171-47d2-8e86-c153a6243c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:17.378287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acff35-0171-47d2-8e86-c153a6243c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.378709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acff35-0171-47d2-8e86-c153a6243c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.378990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1acff35-0171-47d2-8e86-c153a6243c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.379668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1acff35-0171-47d2-8e86-c153a6243c50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:17.381570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1acff35-0171-47d2-8e86-c153a6243c50 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:17.426484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1264/2529] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:17 2022] POST /identity/v3/projects => generated 401 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:17.429127 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6e35e-e9ac-45fa-9b4d-ba4aefae7997 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:17.440138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6e35e-e9ac-45fa-9b4d-ba4aefae7997 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.478023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:17.515538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628297e-d2f5-41ba-b44c-7383fe932759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:17.516478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628297e-d2f5-41ba-b44c-7383fe932759 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.517017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628297e-d2f5-41ba-b44c-7383fe932759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.518037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628297e-d2f5-41ba-b44c-7383fe932759 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.518898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628297e-d2f5-41ba-b44c-7383fe932759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:17.520948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628297e-d2f5-41ba-b44c-7383fe932759 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:17.544695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e2149-a058-4395-9919-04175fe1898f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.544988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e2149-a058-4395-9919-04175fe1898f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.545540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e2149-a058-4395-9919-04175fe1898f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.548724 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1266/2530] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:17 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:17.552896 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628297e-d2f5-41ba-b44c-7383fe932759 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:17.563544 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628297e-d2f5-41ba-b44c-7383fe932759 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.568916 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c275-160d-4832-b2a7-e53bffd6ee6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:17.569532 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c275-160d-4832-b2a7-e53bffd6ee6e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.569813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c275-160d-4832-b2a7-e53bffd6ee6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.570082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c275-160d-4832-b2a7-e53bffd6ee6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.570807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44c275-160d-4832-b2a7-e53bffd6ee6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:17.571011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-096e2149-a058-4395-9919-04175fe1898f None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:17.572405 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44c275-160d-4832-b2a7-e53bffd6ee6e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:17.572535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-096e2149-a058-4395-9919-04175fe1898f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:17.587350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1267/2531] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:04:17.588434 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c275-160d-4832-b2a7-e53bffd6ee6e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:17.592581 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:17.599661 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c275-160d-4832-b2a7-e53bffd6ee6e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.609242 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b792d-3913-4d97-83f9-ced2dc7ff2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:17.610122 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b792d-3913-4d97-83f9-ced2dc7ff2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.610395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b792d-3913-4d97-83f9-ced2dc7ff2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.610737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b792d-3913-4d97-83f9-ced2dc7ff2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.612355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b792d-3913-4d97-83f9-ced2dc7ff2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:17.614157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b792d-3913-4d97-83f9-ced2dc7ff2a5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:17.625502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1268/2532] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:04:17.632435 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b792d-3913-4d97-83f9-ced2dc7ff2a5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:17.642352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b792d-3913-4d97-83f9-ced2dc7ff2a5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.646995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c10d2-4c1e-4fc9-9201-cf2d1e35e355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:17.647644 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c10d2-4c1e-4fc9-9201-cf2d1e35e355 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.648581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c10d2-4c1e-4fc9-9201-cf2d1e35e355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.648956 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c10d2-4c1e-4fc9-9201-cf2d1e35e355 admin admin] PATH_INFO: `/v3/projects/1a13d450cab546018f20ce7cc29223f4/users/6a52e84932a54de693803e98b8c52903/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.652337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1265/2533] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 05:04:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:04:17.660168 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e2149-a058-4395-9919-04175fe1898f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:17.661587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-096e2149-a058-4395-9919-04175fe1898f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:17.664512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c10d2-4c1e-4fc9-9201-cf2d1e35e355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a13d450cab546018f20ce7cc29223f4, user_id=6a52e84932a54de693803e98b8c52903, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:17.667206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c10d2-4c1e-4fc9-9201-cf2d1e35e355 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:17.684400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1269/2534] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:17 2022] PUT /identity/v3/projects/1a13d450cab546018f20ce7cc29223f4/users/6a52e84932a54de693803e98b8c52903/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:04:17.697685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b180b4b-3370-420c-a158-d940b215ff10 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.697991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b180b4b-3370-420c-a158-d940b215ff10 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.698233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b180b4b-3370-420c-a158-d940b215ff10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.729857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4b180b4b-3370-420c-a158-d940b215ff10 None None] MFA Rules not processed for user `6a52e84932a54de693803e98b8c52903`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:17.731212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4b180b4b-3370-420c-a158-d940b215ff10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:17.745682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e2149-a058-4395-9919-04175fe1898f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.815861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24da75-fd56-4f41-84eb-4b7ff0fe6867 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:17.816508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24da75-fd56-4f41-84eb-4b7ff0fe6867 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.816836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24da75-fd56-4f41-84eb-4b7ff0fe6867 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.817110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24da75-fd56-4f41-84eb-4b7ff0fe6867 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.820370 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1270/2535] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:04:17.824895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b180b4b-3370-420c-a158-d940b215ff10 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:17.880167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24da75-fd56-4f41-84eb-4b7ff0fe6867 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:17.881759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24da75-fd56-4f41-84eb-4b7ff0fe6867 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:17.894776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1266/2536] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:04:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 30 05:04:17.896666 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b180b4b-3370-420c-a158-d940b215ff10 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.901440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a1448-c9f8-4e71-a8de-c7a19dcccdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:17.902065 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a1448-c9f8-4e71-a8de-c7a19dcccdce admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:17.902344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a1448-c9f8-4e71-a8de-c7a19dcccdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:17.902649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a1448-c9f8-4e71-a8de-c7a19dcccdce admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:17.903321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a1448-c9f8-4e71-a8de-c7a19dcccdce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:17.905225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a1448-c9f8-4e71-a8de-c7a19dcccdce admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:17.929847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:17.942163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1271/2537] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:17 2022] POST /identity/v3/projects => generated 401 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:04:17.947280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24da75-fd56-4f41-84eb-4b7ff0fe6867 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:17.960364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24da75-fd56-4f41-84eb-4b7ff0fe6867 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:17.965339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:18.039673 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f15d98-a663-4112-ae8e-2db866db775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:18.040325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f15d98-a663-4112-ae8e-2db866db775a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:18.040580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f15d98-a663-4112-ae8e-2db866db775a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:18.040909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f15d98-a663-4112-ae8e-2db866db775a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:18.041558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f15d98-a663-4112-ae8e-2db866db775a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:18.043609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f15d98-a663-4112-ae8e-2db866db775a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:18.083351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1267/2538] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:17 2022] POST /identity/v3/users => generated 307 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:18.087278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a1448-c9f8-4e71-a8de-c7a19dcccdce admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:18.104088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a1448-c9f8-4e71-a8de-c7a19dcccdce admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:18.109795 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea08c-4e29-440c-8d69-9641d4331025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:18.110675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea08c-4e29-440c-8d69-9641d4331025 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:18.111682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea08c-4e29-440c-8d69-9641d4331025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:18.112495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fea08c-4e29-440c-8d69-9641d4331025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:18.113714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fea08c-4e29-440c-8d69-9641d4331025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:18.116986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fea08c-4e29-440c-8d69-9641d4331025 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:18.130554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1272/2539] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:18.135019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f15d98-a663-4112-ae8e-2db866db775a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:18.145832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f15d98-a663-4112-ae8e-2db866db775a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:18.154017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba0777-c54a-407e-8468-4d7d08d03459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:18.154907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba0777-c54a-407e-8468-4d7d08d03459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:18.155191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba0777-c54a-407e-8468-4d7d08d03459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:18.155472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba0777-c54a-407e-8468-4d7d08d03459 admin admin] PATH_INFO: `/v3/projects/b3446e416a9945869e13d7570ea9734c/users/4070c312f3dc463b8f18eaec4e90ce5c/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:18.172796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba0777-c54a-407e-8468-4d7d08d03459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3446e416a9945869e13d7570ea9734c, user_id=4070c312f3dc463b8f18eaec4e90ce5c, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:18.176899 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba0777-c54a-407e-8468-4d7d08d03459 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:18.194940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1268/2540] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:18 2022] PUT /identity/v3/projects/b3446e416a9945869e13d7570ea9734c/users/4070c312f3dc463b8f18eaec4e90ce5c/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:18.198816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea08c-4e29-440c-8d69-9641d4331025 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:18.214130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fea08c-4e29-440c-8d69-9641d4331025 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:18.312303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e572d1-c0fa-4a0f-9300-d92320afc201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:18.313396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e572d1-c0fa-4a0f-9300-d92320afc201 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:18.313396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e572d1-c0fa-4a0f-9300-d92320afc201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:18.313622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e572d1-c0fa-4a0f-9300-d92320afc201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:18.314838 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e572d1-c0fa-4a0f-9300-d92320afc201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:18.318408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e572d1-c0fa-4a0f-9300-d92320afc201 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:18.335605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1273/2541] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:18.339562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba0777-c54a-407e-8468-4d7d08d03459 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:18.352111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba0777-c54a-407e-8468-4d7d08d03459 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:18.358651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e2a06-bbbb-4cc9-9500-76af3a18b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:18.359661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2a06-bbbb-4cc9-9500-76af3a18b0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:18.360149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2a06-bbbb-4cc9-9500-76af3a18b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:18.360613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e2a06-bbbb-4cc9-9500-76af3a18b0e8 admin admin] PATH_INFO: `/v3/projects/b3446e416a9945869e13d7570ea9734c/users/4070c312f3dc463b8f18eaec4e90ce5c/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:18.363610 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e2a06-bbbb-4cc9-9500-76af3a18b0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3446e416a9945869e13d7570ea9734c, user_id=4070c312f3dc463b8f18eaec4e90ce5c, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:18.368367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e2a06-bbbb-4cc9-9500-76af3a18b0e8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:18.383892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1269/2542] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:18 2022] PUT /identity/v3/projects/b3446e416a9945869e13d7570ea9734c/users/4070c312f3dc463b8f18eaec4e90ce5c/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:18.397114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe748de-8d4a-42cc-acb9-9c4d847c9b64 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:18.397499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe748de-8d4a-42cc-acb9-9c4d847c9b64 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:18.397844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe748de-8d4a-42cc-acb9-9c4d847c9b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:18.444154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3fe748de-8d4a-42cc-acb9-9c4d847c9b64 None None] MFA Rules not processed for user `4070c312f3dc463b8f18eaec4e90ce5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:18.445739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3fe748de-8d4a-42cc-acb9-9c4d847c9b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:18.552811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1274/2543] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:18.564619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e2a06-bbbb-4cc9-9500-76af3a18b0e8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:18.567199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fd4e2a06-bbbb-4cc9-9500-76af3a18b0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:18.634772 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e2a06-bbbb-4cc9-9500-76af3a18b0e8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:18.638866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec149910-8c68-4d11-99c1-270ac626d170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:18.639495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec149910-8c68-4d11-99c1-270ac626d170 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:18.639777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec149910-8c68-4d11-99c1-270ac626d170 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:18.640052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec149910-8c68-4d11-99c1-270ac626d170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:18.651530 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec149910-8c68-4d11-99c1-270ac626d170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:18.653041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec149910-8c68-4d11-99c1-270ac626d170 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:18.665963 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1270/2544] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:18 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:04:18.779460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe748de-8d4a-42cc-acb9-9c4d847c9b64 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:18.790925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe748de-8d4a-42cc-acb9-9c4d847c9b64 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:18.794989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9650ed-f899-44b2-86ea-170f74a05bf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:18.798198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9650ed-f899-44b2-86ea-170f74a05bf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:18.798198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9650ed-f899-44b2-86ea-170f74a05bf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:18.798198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9650ed-f899-44b2-86ea-170f74a05bf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:18.876062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9650ed-f899-44b2-86ea-170f74a05bf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:18.878422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9650ed-f899-44b2-86ea-170f74a05bf3 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:18.885341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:18.891424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1275/2545] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:19.347854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec149910-8c68-4d11-99c1-270ac626d170 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:19.349373 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec149910-8c68-4d11-99c1-270ac626d170 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:19.447483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:19.456337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec149910-8c68-4d11-99c1-270ac626d170 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:19.463929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c2d6b-407c-480a-a81c-87e9bc0b7d6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:19.464681 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c2d6b-407c-480a-a81c-87e9bc0b7d6e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:19.465027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c2d6b-407c-480a-a81c-87e9bc0b7d6e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:19.465365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c2d6b-407c-480a-a81c-87e9bc0b7d6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:19.467245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-553c2d6b-407c-480a-a81c-87e9bc0b7d6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:19.483853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c2d6b-407c-480a-a81c-87e9bc0b7d6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:19.486500 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c2d6b-407c-480a-a81c-87e9bc0b7d6e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:19.500250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1271/2546] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:23.285978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9650ed-f899-44b2-86ea-170f74a05bf3 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:23.369052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9650ed-f899-44b2-86ea-170f74a05bf3 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:23.374852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:23.375537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:23.375816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:23.376089 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] PATH_INFO: `/v3/users/6a52e84932a54de693803e98b8c52903` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:23.378204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a52e84932a54de693803e98b8c52903)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:23.380379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:23.447987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a52e84932a54de693803e98b8c52903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:23.457317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a52e84932a54de693803e98b8c52903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:23.458089 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a52e84932a54de693803e98b8c52903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:23.458747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a52e84932a54de693803e98b8c52903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:23.473540 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a52e84932a54de693803e98b8c52903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:23.491224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1276/2547] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:23 2022] DELETE /identity/v3/users/6a52e84932a54de693803e98b8c52903 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:23.495632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c68cb-b89d-4e94-8243-a63d3c8f6b7f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:23.496175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c68cb-b89d-4e94-8243-a63d3c8f6b7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:23.496290 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c68cb-b89d-4e94-8243-a63d3c8f6b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:23.517723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d48c68cb-b89d-4e94-8243-a63d3c8f6b7f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:23.519018 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d48c68cb-b89d-4e94-8243-a63d3c8f6b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:23.597260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1272/2548] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:04:23.607106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:23.608515 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:23.683523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670c6d2-1281-491f-91cb-fd39e7bbda6f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:23.688116 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cd88b-8119-4c09-83d4-8b7410284722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:23.688822 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cd88b-8119-4c09-83d4-8b7410284722 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:23.689293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cd88b-8119-4c09-83d4-8b7410284722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:23.689591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cd88b-8119-4c09-83d4-8b7410284722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:23.702574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cd88b-8119-4c09-83d4-8b7410284722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:23.704061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cd88b-8119-4c09-83d4-8b7410284722 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:23.714722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1277/2549] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:24.125660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c68cb-b89d-4e94-8243-a63d3c8f6b7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:24.127401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d48c68cb-b89d-4e94-8243-a63d3c8f6b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:24.153407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c68cb-b89d-4e94-8243-a63d3c8f6b7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:24.179488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:24.180711 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:24.181419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:24.181806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] PATH_INFO: `/v3/projects/1a13d450cab546018f20ce7cc29223f4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:24.183612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a13d450cab546018f20ce7cc29223f4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:24.185811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:24.250853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a13d450cab546018f20ce7cc29223f4', 'request_id': 'req-a80b39e7-eccf-405c-b09e-44648a0d1138'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.263523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a13d450cab546018f20ce7cc29223f4', 'request_id': 'req-a80b39e7-eccf-405c-b09e-44648a0d1138'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.270823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a13d450cab546018f20ce7cc29223f4', 'request_id': 'req-a80b39e7-eccf-405c-b09e-44648a0d1138'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.285402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] The token cache is being invalidate because project 1a13d450cab546018f20ce7cc29223f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:24.285707 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.286353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.291028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1273/2550] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:24 2022] DELETE /identity/v3/projects/1a13d450cab546018f20ce7cc29223f4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:24.292535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cd88b-8119-4c09-83d4-8b7410284722 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:24.304382 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cd88b-8119-4c09-83d4-8b7410284722 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:24.328616 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:24.383678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:24.384344 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:24.384610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:24.384883 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] PATH_INFO: `/v3/users/4070c312f3dc463b8f18eaec4e90ce5c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:24.386155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4070c312f3dc463b8f18eaec4e90ce5c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:24.388017 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:24.446310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4070c312f3dc463b8f18eaec4e90ce5c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.454983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4070c312f3dc463b8f18eaec4e90ce5c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.456023 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4070c312f3dc463b8f18eaec4e90ce5c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.456776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4070c312f3dc463b8f18eaec4e90ce5c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.472828 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffa42aea-bcd2-4ffd-b573-fc2a76203536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4070c312f3dc463b8f18eaec4e90ce5c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:24.516648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1278/2551] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:24 2022] DELETE /identity/v3/users/4070c312f3dc463b8f18eaec4e90ce5c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:24.869287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:24.870800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:24.893439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b39e7-eccf-405c-b09e-44648a0d1138 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:24.973445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:24.974100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:24.974380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:24.974683 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] PATH_INFO: `/v3/projects/b3446e416a9945869e13d7570ea9734c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:24.975891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3446e416a9945869e13d7570ea9734c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:24.977798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:24.995555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:25.040864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3446e416a9945869e13d7570ea9734c', 'request_id': 'req-5aa69a13-332c-4e68-810f-555fb90c7207'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:25.051092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3446e416a9945869e13d7570ea9734c', 'request_id': 'req-5aa69a13-332c-4e68-810f-555fb90c7207'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:25.057937 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3446e416a9945869e13d7570ea9734c', 'request_id': 'req-5aa69a13-332c-4e68-810f-555fb90c7207'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:25.073841 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] The token cache is being invalidate because project b3446e416a9945869e13d7570ea9734c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:25.074134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:25.074970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5aa69a13-332c-4e68-810f-555fb90c7207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:25.076552 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1274/2552] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:24 2022] DELETE /identity/v3/projects/b3446e416a9945869e13d7570ea9734c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:25.082288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a863d-1d2e-4e56-9037-1a0fd7dff1ea None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.082624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a863d-1d2e-4e56-9037-1a0fd7dff1ea None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.082893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a863d-1d2e-4e56-9037-1a0fd7dff1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.103885 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-469a863d-1d2e-4e56-9037-1a0fd7dff1ea None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:25.105598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-469a863d-1d2e-4e56-9037-1a0fd7dff1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:25.178665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1279/2553] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:25.186240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06967320-399f-459b-afab-a0461302b5f1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.186618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06967320-399f-459b-afab-a0461302b5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.186962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06967320-399f-459b-afab-a0461302b5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.210920 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-06967320-399f-459b-afab-a0461302b5f1 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:25.212363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06967320-399f-459b-afab-a0461302b5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:25.221227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1275/2554] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:25.225333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a863d-1d2e-4e56-9037-1a0fd7dff1ea None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:25.237388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a863d-1d2e-4e56-9037-1a0fd7dff1ea None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:25.242625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc187e8-473f-47d9-b8fb-9bb684a8d0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:25.243278 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc187e8-473f-47d9-b8fb-9bb684a8d0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.243534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc187e8-473f-47d9-b8fb-9bb684a8d0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.243803 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc187e8-473f-47d9-b8fb-9bb684a8d0a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.244805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc187e8-473f-47d9-b8fb-9bb684a8d0a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:04:25.245105 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc187e8-473f-47d9-b8fb-9bb684a8d0a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:25.246745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc187e8-473f-47d9-b8fb-9bb684a8d0a3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:25.264098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1280/2555] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:04:25.265587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06967320-399f-459b-afab-a0461302b5f1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:25.275800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06967320-399f-459b-afab-a0461302b5f1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:25.281253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f9691-5be4-47e9-82af-c17a4ec444f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:25.281916 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9691-5be4-47e9-82af-c17a4ec444f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.282199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9691-5be4-47e9-82af-c17a4ec444f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.282601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9691-5be4-47e9-82af-c17a4ec444f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.283402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f9691-5be4-47e9-82af-c17a4ec444f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:25.286292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f9691-5be4-47e9-82af-c17a4ec444f4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:25.346463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1276/2556] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:25 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:25.351570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc187e8-473f-47d9-b8fb-9bb684a8d0a3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:25.367847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc187e8-473f-47d9-b8fb-9bb684a8d0a3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:25.413073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:25.464751 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22fda8-d99b-4346-92cd-f6b3f381c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:25.465696 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22fda8-d99b-4346-92cd-f6b3f381c7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.466038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22fda8-d99b-4346-92cd-f6b3f381c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.466393 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22fda8-d99b-4346-92cd-f6b3f381c7e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.467229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22fda8-d99b-4346-92cd-f6b3f381c7e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:25.469261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22fda8-d99b-4346-92cd-f6b3f381c7e6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:25.498690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1281/2557] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:25 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:25.504158 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f9691-5be4-47e9-82af-c17a4ec444f4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:25.522256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f9691-5be4-47e9-82af-c17a4ec444f4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:25.539851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536ce83-c714-40e9-9454-321dd9a4cb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:25.540656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536ce83-c714-40e9-9454-321dd9a4cb78 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.540997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536ce83-c714-40e9-9454-321dd9a4cb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.541359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536ce83-c714-40e9-9454-321dd9a4cb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.542139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536ce83-c714-40e9-9454-321dd9a4cb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:25.543970 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536ce83-c714-40e9-9454-321dd9a4cb78 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:25.556960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1277/2558] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:04:25.561120 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22fda8-d99b-4346-92cd-f6b3f381c7e6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:25.572325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22fda8-d99b-4346-92cd-f6b3f381c7e6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:25.577160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f5997-0235-43e3-bee7-bcf5b83cb0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:25.577781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5997-0235-43e3-bee7-bcf5b83cb0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.578063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5997-0235-43e3-bee7-bcf5b83cb0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.578407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f5997-0235-43e3-bee7-bcf5b83cb0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.579150 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5997-0235-43e3-bee7-bcf5b83cb0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:25.581325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f5997-0235-43e3-bee7-bcf5b83cb0c4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:25.594476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1282/2559] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:25.597621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536ce83-c714-40e9-9454-321dd9a4cb78 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:25.608113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536ce83-c714-40e9-9454-321dd9a4cb78 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:25.613908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac092b-bd2d-4d5b-8d0e-af51ad645702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:25.614651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac092b-bd2d-4d5b-8d0e-af51ad645702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.614931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac092b-bd2d-4d5b-8d0e-af51ad645702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.615207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac092b-bd2d-4d5b-8d0e-af51ad645702 admin admin] PATH_INFO: `/v3/projects/261975d5c2d44a9abb134e44a1ff9594/users/77be2606ae3a4335a1363413cfe4d1a6/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.635525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac092b-bd2d-4d5b-8d0e-af51ad645702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=261975d5c2d44a9abb134e44a1ff9594, user_id=77be2606ae3a4335a1363413cfe4d1a6, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:25.639105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac092b-bd2d-4d5b-8d0e-af51ad645702 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:25.668719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1278/2560] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:25 2022] PUT /identity/v3/projects/261975d5c2d44a9abb134e44a1ff9594/users/77be2606ae3a4335a1363413cfe4d1a6/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:25.684366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e136d-6e0f-403e-adc5-2d25d337081a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.684665 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e136d-6e0f-403e-adc5-2d25d337081a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.685057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e136d-6e0f-403e-adc5-2d25d337081a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.718181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-785e136d-6e0f-403e-adc5-2d25d337081a None None] MFA Rules not processed for user `77be2606ae3a4335a1363413cfe4d1a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:25.719517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-785e136d-6e0f-403e-adc5-2d25d337081a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:25.814484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1283/2561] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:25.816396 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac092b-bd2d-4d5b-8d0e-af51ad645702 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:25.908291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac092b-bd2d-4d5b-8d0e-af51ad645702 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:25.913363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4687dfa-0a6c-4ddf-b240-fbccbe80626e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:25.914163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4687dfa-0a6c-4ddf-b240-fbccbe80626e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:25.914560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4687dfa-0a6c-4ddf-b240-fbccbe80626e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:25.914837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4687dfa-0a6c-4ddf-b240-fbccbe80626e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:25.915678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4687dfa-0a6c-4ddf-b240-fbccbe80626e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:25.917606 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4687dfa-0a6c-4ddf-b240-fbccbe80626e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:25.950807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1279/2562] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:25 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:25.955271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e136d-6e0f-403e-adc5-2d25d337081a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:25.967227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e136d-6e0f-403e-adc5-2d25d337081a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:26.042379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a82f4-8867-4015-bd35-2b0fed30ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:26.043225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a82f4-8867-4015-bd35-2b0fed30ebbd admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:26.043571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a82f4-8867-4015-bd35-2b0fed30ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:26.043854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a82f4-8867-4015-bd35-2b0fed30ebbd admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:26.044536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a82f4-8867-4015-bd35-2b0fed30ebbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:26.046744 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a82f4-8867-4015-bd35-2b0fed30ebbd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:26.085639 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1284/2563] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:25 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:04:26.086766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4687dfa-0a6c-4ddf-b240-fbccbe80626e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:26.105393 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4687dfa-0a6c-4ddf-b240-fbccbe80626e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:26.116194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cae96c-1c67-4e9c-b124-6c9e9535e8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:26.116837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cae96c-1c67-4e9c-b124-6c9e9535e8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:26.117119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cae96c-1c67-4e9c-b124-6c9e9535e8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:26.117394 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cae96c-1c67-4e9c-b124-6c9e9535e8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:26.118101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cae96c-1c67-4e9c-b124-6c9e9535e8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:26.119905 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cae96c-1c67-4e9c-b124-6c9e9535e8b6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:26.136948 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1280/2564] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:26.138371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a82f4-8867-4015-bd35-2b0fed30ebbd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:26.164268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a82f4-8867-4015-bd35-2b0fed30ebbd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:26.175949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135a00f-ec7a-474c-9bef-4a5b17b6b4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:26.176851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135a00f-ec7a-474c-9bef-4a5b17b6b4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:26.177131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135a00f-ec7a-474c-9bef-4a5b17b6b4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:26.177404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135a00f-ec7a-474c-9bef-4a5b17b6b4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:26.178110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135a00f-ec7a-474c-9bef-4a5b17b6b4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:26.179768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135a00f-ec7a-474c-9bef-4a5b17b6b4e8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:26.200865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1285/2565] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:26.205142 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cae96c-1c67-4e9c-b124-6c9e9535e8b6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:26.220821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cae96c-1c67-4e9c-b124-6c9e9535e8b6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:26.226164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa476628-77dd-4263-a08d-a5f23e516c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:26.227009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476628-77dd-4263-a08d-a5f23e516c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:26.227272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476628-77dd-4263-a08d-a5f23e516c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:26.227574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476628-77dd-4263-a08d-a5f23e516c19 admin admin] PATH_INFO: `/v3/projects/efa11ed5a51644d38eebabf813f91ade/users/9a52e73c0727447c89db2acf3673a80f/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:26.243132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:26.248002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa476628-77dd-4263-a08d-a5f23e516c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efa11ed5a51644d38eebabf813f91ade, user_id=9a52e73c0727447c89db2acf3673a80f, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:26.251923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa476628-77dd-4263-a08d-a5f23e516c19 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:26.270786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1281/2566] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:26 2022] PUT /identity/v3/projects/efa11ed5a51644d38eebabf813f91ade/users/9a52e73c0727447c89db2acf3673a80f/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:26.281257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a036b7-fa02-428e-a649-4ed9f466349c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:26.281895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a036b7-fa02-428e-a649-4ed9f466349c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:26.282948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a036b7-fa02-428e-a649-4ed9f466349c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:26.317005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a8a036b7-fa02-428e-a649-4ed9f466349c None None] MFA Rules not processed for user `9a52e73c0727447c89db2acf3673a80f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:26.318648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a8a036b7-fa02-428e-a649-4ed9f466349c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:26.322874 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:26.403796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1286/2567] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:26.408819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa476628-77dd-4263-a08d-a5f23e516c19 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:26.487950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa476628-77dd-4263-a08d-a5f23e516c19 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:26.493546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f627a-07f5-49f9-955e-bc817cf2c788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:26.494204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f627a-07f5-49f9-955e-bc817cf2c788 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:26.494686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f627a-07f5-49f9-955e-bc817cf2c788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:26.494967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f627a-07f5-49f9-955e-bc817cf2c788 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:26.495650 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f627a-07f5-49f9-955e-bc817cf2c788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:26.497946 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f627a-07f5-49f9-955e-bc817cf2c788 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:26.545545 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1282/2568] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:26 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:26.547097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a036b7-fa02-428e-a649-4ed9f466349c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:26.564903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a036b7-fa02-428e-a649-4ed9f466349c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:26.634095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebb7fe-6253-4932-aa65-5f83e3ac239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:26.635082 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebb7fe-6253-4932-aa65-5f83e3ac239e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:26.635362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebb7fe-6253-4932-aa65-5f83e3ac239e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:26.635636 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebb7fe-6253-4932-aa65-5f83e3ac239e admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:26.636318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebb7fe-6253-4932-aa65-5f83e3ac239e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:26.638294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebb7fe-6253-4932-aa65-5f83e3ac239e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:26.690503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1287/2569] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:26 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:26.690745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f627a-07f5-49f9-955e-bc817cf2c788 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:26.716924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f627a-07f5-49f9-955e-bc817cf2c788 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:26.727742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7c221-7035-4d4e-9ac8-6b5810542e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:26.739068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7c221-7035-4d4e-9ac8-6b5810542e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:26.739068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7c221-7035-4d4e-9ac8-6b5810542e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:26.739068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7c221-7035-4d4e-9ac8-6b5810542e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:26.739068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7c221-7035-4d4e-9ac8-6b5810542e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:26.739068 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7c221-7035-4d4e-9ac8-6b5810542e8a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:26.805719 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1283/2570] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:26.830538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebb7fe-6253-4932-aa65-5f83e3ac239e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:26.858726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebb7fe-6253-4932-aa65-5f83e3ac239e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:26.870424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86188fe-eb55-41e2-8408-218f6d92bc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:26.871167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86188fe-eb55-41e2-8408-218f6d92bc35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:26.871499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86188fe-eb55-41e2-8408-218f6d92bc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:26.871926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86188fe-eb55-41e2-8408-218f6d92bc35 admin admin] PATH_INFO: `/v3/projects/f0d1ca480689490b82a028d1fe66dc2f/users/8e3186a1ea2e4fc38741e2fcf25a9c9f/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:26.893352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86188fe-eb55-41e2-8408-218f6d92bc35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d1ca480689490b82a028d1fe66dc2f, user_id=8e3186a1ea2e4fc38741e2fcf25a9c9f, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:26.897119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86188fe-eb55-41e2-8408-218f6d92bc35 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:26.922599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1288/2571] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:26 2022] PUT /identity/v3/projects/f0d1ca480689490b82a028d1fe66dc2f/users/8e3186a1ea2e4fc38741e2fcf25a9c9f/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:26.925678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7c221-7035-4d4e-9ac8-6b5810542e8a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:26.937643 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7c221-7035-4d4e-9ac8-6b5810542e8a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:27.060347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5cc78-48a3-4373-8862-b0a3851184ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:27.061277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5cc78-48a3-4373-8862-b0a3851184ce admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:27.061563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5cc78-48a3-4373-8862-b0a3851184ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:27.061886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5cc78-48a3-4373-8862-b0a3851184ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:27.062637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5cc78-48a3-4373-8862-b0a3851184ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:27.065845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5cc78-48a3-4373-8862-b0a3851184ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:27.085807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1284/2572] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:27.086048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86188fe-eb55-41e2-8408-218f6d92bc35 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:27.100334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86188fe-eb55-41e2-8408-218f6d92bc35 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:27.106130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3355008-93aa-4fdf-b928-d1df0fda3420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:27.106911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3355008-93aa-4fdf-b928-d1df0fda3420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:27.107288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3355008-93aa-4fdf-b928-d1df0fda3420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:27.107578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3355008-93aa-4fdf-b928-d1df0fda3420 admin admin] PATH_INFO: `/v3/projects/f0d1ca480689490b82a028d1fe66dc2f/users/8e3186a1ea2e4fc38741e2fcf25a9c9f/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:27.109758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3355008-93aa-4fdf-b928-d1df0fda3420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d1ca480689490b82a028d1fe66dc2f, user_id=8e3186a1ea2e4fc38741e2fcf25a9c9f, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:27.113269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3355008-93aa-4fdf-b928-d1df0fda3420 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:27.129119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1289/2573] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:27 2022] PUT /identity/v3/projects/f0d1ca480689490b82a028d1fe66dc2f/users/8e3186a1ea2e4fc38741e2fcf25a9c9f/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:04:27.146583 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c444ce4-91c8-439f-b8a6-a43b3f59c93f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:27.146853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c444ce4-91c8-439f-b8a6-a43b3f59c93f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:27.147130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c444ce4-91c8-439f-b8a6-a43b3f59c93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:27.194869 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9c444ce4-91c8-439f-b8a6-a43b3f59c93f None None] MFA Rules not processed for user `8e3186a1ea2e4fc38741e2fcf25a9c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:27.196482 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c444ce4-91c8-439f-b8a6-a43b3f59c93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:27.295947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:27.328257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1285/2574] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:27.346802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3355008-93aa-4fdf-b928-d1df0fda3420 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:27.348675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b3355008-93aa-4fdf-b928-d1df0fda3420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:27.424538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3355008-93aa-4fdf-b928-d1df0fda3420 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:27.434380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96080805-b14c-463b-ad95-ec686af22fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:27.437249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96080805-b14c-463b-ad95-ec686af22fba service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:27.437249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96080805-b14c-463b-ad95-ec686af22fba service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:27.437249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96080805-b14c-463b-ad95-ec686af22fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:27.449283 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96080805-b14c-463b-ad95-ec686af22fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:27.452142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96080805-b14c-463b-ad95-ec686af22fba service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:27.458871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:27.467027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1290/2575] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:27.557504 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c444ce4-91c8-439f-b8a6-a43b3f59c93f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:27.559191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c444ce4-91c8-439f-b8a6-a43b3f59c93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:27.573149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c444ce4-91c8-439f-b8a6-a43b3f59c93f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:27.584351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe97d3-7f50-4e5b-b21b-b76d4dc9e7db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:27.585252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe97d3-7f50-4e5b-b21b-b76d4dc9e7db service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:27.585702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe97d3-7f50-4e5b-b21b-b76d4dc9e7db service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:27.586150 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe97d3-7f50-4e5b-b21b-b76d4dc9e7db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:27.707235 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe97d3-7f50-4e5b-b21b-b76d4dc9e7db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:27.708982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe97d3-7f50-4e5b-b21b-b76d4dc9e7db service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:27.728831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1286/2576] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:29.262670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eadaf-ec48-4b44-b195-39339e09b3de None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:29.262670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eadaf-ec48-4b44-b195-39339e09b3de None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:29.262670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6eadaf-ec48-4b44-b195-39339e09b3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:29.280712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-db6eadaf-ec48-4b44-b195-39339e09b3de None None] MFA Rules not processed for user `407ad282edd74fb1adf62fb63306779c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:29.281918 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-db6eadaf-ec48-4b44-b195-39339e09b3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:29.349504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1291/2577] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:29.727444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe97d3-7f50-4e5b-b21b-b76d4dc9e7db service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:29.728950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5efe97d3-7f50-4e5b-b21b-b76d4dc9e7db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:29.793865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe97d3-7f50-4e5b-b21b-b76d4dc9e7db service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:29.797676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d270a8da-07ea-4612-b69e-0a276f70464b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:29.798476 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270a8da-07ea-4612-b69e-0a276f70464b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:29.798760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270a8da-07ea-4612-b69e-0a276f70464b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:29.799031 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270a8da-07ea-4612-b69e-0a276f70464b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:29.800745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d270a8da-07ea-4612-b69e-0a276f70464b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:29.810124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270a8da-07ea-4612-b69e-0a276f70464b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:29.811875 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d270a8da-07ea-4612-b69e-0a276f70464b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:29.820101 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1287/2578] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:30.893615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eadaf-ec48-4b44-b195-39339e09b3de None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:30.955766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6eadaf-ec48-4b44-b195-39339e09b3de None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:30.960376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:30.961042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:30.961320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:30.961591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] PATH_INFO: `/v3/users/77be2606ae3a4335a1363413cfe4d1a6` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:30.962794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=77be2606ae3a4335a1363413cfe4d1a6)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:30.964750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:31.012707 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77be2606ae3a4335a1363413cfe4d1a6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.020030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77be2606ae3a4335a1363413cfe4d1a6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.020710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77be2606ae3a4335a1363413cfe4d1a6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.021313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77be2606ae3a4335a1363413cfe4d1a6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.033845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77be2606ae3a4335a1363413cfe4d1a6'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.047493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1292/2579] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:30 2022] DELETE /identity/v3/users/77be2606ae3a4335a1363413cfe4d1a6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:31.052578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e233e7-98b1-4ecf-9d51-4f8b557268ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:31.053077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e233e7-98b1-4ecf-9d51-4f8b557268ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:31.053172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e233e7-98b1-4ecf-9d51-4f8b557268ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:31.071788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-65e233e7-98b1-4ecf-9d51-4f8b557268ef None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:31.072985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65e233e7-98b1-4ecf-9d51-4f8b557268ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:31.131191 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1288/2580] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:31.140853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:31.143743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:31.176238 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:31.217869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f8989-0fab-4a48-97e5-8459f958ddcc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:31.224949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030f936-a6d9-43a6-8bd9-629c702d7771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:31.225815 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030f936-a6d9-43a6-8bd9-629c702d7771 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:31.226091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030f936-a6d9-43a6-8bd9-629c702d7771 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:31.226523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030f936-a6d9-43a6-8bd9-629c702d7771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:31.237021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030f936-a6d9-43a6-8bd9-629c702d7771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:31.238743 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030f936-a6d9-43a6-8bd9-629c702d7771 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:31.248939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1293/2581] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:31.548945 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e233e7-98b1-4ecf-9d51-4f8b557268ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:31.550686 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65e233e7-98b1-4ecf-9d51-4f8b557268ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:31.565602 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e233e7-98b1-4ecf-9d51-4f8b557268ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:31.571304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:31.571958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:31.572232 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:31.572499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] PATH_INFO: `/v3/projects/261975d5c2d44a9abb134e44a1ff9594` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:31.573694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=261975d5c2d44a9abb134e44a1ff9594)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:31.575635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:31.600307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:31.623646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '261975d5c2d44a9abb134e44a1ff9594', 'request_id': 'req-d6d2e898-6f8c-495e-920c-b14dee5edaf7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.630145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '261975d5c2d44a9abb134e44a1ff9594', 'request_id': 'req-d6d2e898-6f8c-495e-920c-b14dee5edaf7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.635879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '261975d5c2d44a9abb134e44a1ff9594', 'request_id': 'req-d6d2e898-6f8c-495e-920c-b14dee5edaf7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.647067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] The token cache is being invalidate because project 261975d5c2d44a9abb134e44a1ff9594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:31.647361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.647969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.649443 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1289/2582] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:31 2022] DELETE /identity/v3/projects/261975d5c2d44a9abb134e44a1ff9594 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:31.653476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030f936-a6d9-43a6-8bd9-629c702d7771 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:31.663833 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030f936-a6d9-43a6-8bd9-629c702d7771 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:31.718894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:31.720076 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:31.720324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:31.720594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] PATH_INFO: `/v3/users/9a52e73c0727447c89db2acf3673a80f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:31.721740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a52e73c0727447c89db2acf3673a80f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:31.723727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:31.770602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a52e73c0727447c89db2acf3673a80f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.777287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a52e73c0727447c89db2acf3673a80f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.777934 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a52e73c0727447c89db2acf3673a80f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.778571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a52e73c0727447c89db2acf3673a80f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.790833 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a52e73c0727447c89db2acf3673a80f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:31.804212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1294/2583] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:31 2022] DELETE /identity/v3/users/9a52e73c0727447c89db2acf3673a80f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:32.288682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:32.290233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:32.307553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2e898-6f8c-495e-920c-b14dee5edaf7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:32.380944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:32.383932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:32.384390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:32.384681 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] PATH_INFO: `/v3/projects/efa11ed5a51644d38eebabf813f91ade` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:32.385981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efa11ed5a51644d38eebabf813f91ade)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:32.389057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:32.457798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efa11ed5a51644d38eebabf813f91ade', 'request_id': 'req-c3bf597d-e605-4735-b538-22e8bc7201f0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.476386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efa11ed5a51644d38eebabf813f91ade', 'request_id': 'req-c3bf597d-e605-4735-b538-22e8bc7201f0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.484508 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efa11ed5a51644d38eebabf813f91ade', 'request_id': 'req-c3bf597d-e605-4735-b538-22e8bc7201f0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.499786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] The token cache is being invalidate because project efa11ed5a51644d38eebabf813f91ade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:32.500460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.502318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3bf597d-e605-4735-b538-22e8bc7201f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.504341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1290/2584] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:32 2022] DELETE /identity/v3/projects/efa11ed5a51644d38eebabf813f91ade => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:32.509354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:32.512144 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:32.526760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352e9d-aebe-443a-a810-0d655b641470 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:32.598502 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:32.623701 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:32.624443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:32.625360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:32.626177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] PATH_INFO: `/v3/users/8e3186a1ea2e4fc38741e2fcf25a9c9f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:32.629576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3186a1ea2e4fc38741e2fcf25a9c9f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:32.631714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:32.711380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3186a1ea2e4fc38741e2fcf25a9c9f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.729842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3186a1ea2e4fc38741e2fcf25a9c9f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.731048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3186a1ea2e4fc38741e2fcf25a9c9f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.731829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3186a1ea2e4fc38741e2fcf25a9c9f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.754038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-85884da1-259f-4e16-9cd5-38d39f4a6235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3186a1ea2e4fc38741e2fcf25a9c9f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:32.779226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1295/2585] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:32 2022] DELETE /identity/v3/users/8e3186a1ea2e4fc38741e2fcf25a9c9f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:04:33.357649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448862a-870a-4b63-84f7-71c6a1176559 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:33.358517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448862a-870a-4b63-84f7-71c6a1176559 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:33.359170 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448862a-870a-4b63-84f7-71c6a1176559 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:33.362390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1291/2586] 10.222.0.20 () {54 vars in 921 bytes} [Sat Jul 30 05:04:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 30 05:04:33.371583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f8de4-a0e0-4332-8552-443c57e67105 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:33.371869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f8de4-a0e0-4332-8552-443c57e67105 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:33.372146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f8de4-a0e0-4332-8552-443c57e67105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:33.405702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-322f8de4-a0e0-4332-8552-443c57e67105 None None] MFA Rules not processed for user `a1dc3ad467d9496cb3e0086a0cd8215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:33.408098 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-322f8de4-a0e0-4332-8552-443c57e67105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:33.517151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448862a-870a-4b63-84f7-71c6a1176559 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:33.521037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e448862a-870a-4b63-84f7-71c6a1176559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:33.537813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1296/2587] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Jul 30 05:04:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:33.552809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f8de4-a0e0-4332-8552-443c57e67105 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:33.554562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-322f8de4-a0e0-4332-8552-443c57e67105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:33.637442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:33.670635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448862a-870a-4b63-84f7-71c6a1176559 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:33.676600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:33.677406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:33.677699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:33.678159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] PATH_INFO: `/v3/projects/f0d1ca480689490b82a028d1fe66dc2f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:33.679639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d1ca480689490b82a028d1fe66dc2f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:33.680414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f8de4-a0e0-4332-8552-443c57e67105 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:33.681559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:33.685063 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a033271-800e-453f-8b52-c149f26c9b9d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '548acf58bf9140ec9f1ba126e10deb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:33.685930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a033271-800e-453f-8b52-c149f26c9b9d service glance] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:33.686308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a033271-800e-453f-8b52-c149f26c9b9d service glance] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:33.686909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a033271-800e-453f-8b52-c149f26c9b9d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:33.701978 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a033271-800e-453f-8b52-c149f26c9b9d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:33.703781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a033271-800e-453f-8b52-c149f26c9b9d service glance] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:33.711732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:33.719061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1297/2588] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:33.742795 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d1ca480689490b82a028d1fe66dc2f', 'request_id': 'req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:33.752073 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d1ca480689490b82a028d1fe66dc2f', 'request_id': 'req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:33.759347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d1ca480689490b82a028d1fe66dc2f', 'request_id': 'req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:33.771854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] The token cache is being invalidate because project f0d1ca480689490b82a028d1fe66dc2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:33.772199 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:33.772879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9086cf9-9b9f-4ac8-9559-1a9b208fe3e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:33.774430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1292/2589] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:33 2022] DELETE /identity/v3/projects/f0d1ca480689490b82a028d1fe66dc2f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:33.781167 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3f1b8-53fb-403c-918e-4a83431f6487 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:33.781500 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3f1b8-53fb-403c-918e-4a83431f6487 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:33.781986 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3f1b8-53fb-403c-918e-4a83431f6487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:33.804337 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2da3f1b8-53fb-403c-918e-4a83431f6487 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:33.805534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2da3f1b8-53fb-403c-918e-4a83431f6487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:33.873712 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1298/2590] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:33.882089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fd10b-79bf-410c-9548-cdd5895a2b29 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:33.882374 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fd10b-79bf-410c-9548-cdd5895a2b29 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:33.882685 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fd10b-79bf-410c-9548-cdd5895a2b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:33.903863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b61fd10b-79bf-410c-9548-cdd5895a2b29 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:33.905588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b61fd10b-79bf-410c-9548-cdd5895a2b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:33.914883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1293/2591] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:33.919149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3f1b8-53fb-403c-918e-4a83431f6487 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:33.930548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3f1b8-53fb-403c-918e-4a83431f6487 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:33.935870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7a1f5-a793-4fc8-9b8f-da8b24c82874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:33.936495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7a1f5-a793-4fc8-9b8f-da8b24c82874 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:33.936777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7a1f5-a793-4fc8-9b8f-da8b24c82874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:33.937052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7a1f5-a793-4fc8-9b8f-da8b24c82874 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:33.937754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7a1f5-a793-4fc8-9b8f-da8b24c82874 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:04:33.938061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7a1f5-a793-4fc8-9b8f-da8b24c82874 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:33.940037 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7a1f5-a793-4fc8-9b8f-da8b24c82874 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:33.956249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1299/2592] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 30 05:04:33.957359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fd10b-79bf-410c-9548-cdd5895a2b29 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:33.966625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fd10b-79bf-410c-9548-cdd5895a2b29 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:33.972094 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f38c4-5df8-4423-9a43-dc3993d53012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:33.972705 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f38c4-5df8-4423-9a43-dc3993d53012 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:33.972983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f38c4-5df8-4423-9a43-dc3993d53012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:33.973253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f38c4-5df8-4423-9a43-dc3993d53012 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:33.973927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f38c4-5df8-4423-9a43-dc3993d53012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:33.976501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f38c4-5df8-4423-9a43-dc3993d53012 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:34.007013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1294/2593] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:33 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:34.011003 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7a1f5-a793-4fc8-9b8f-da8b24c82874 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:34.022754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7a1f5-a793-4fc8-9b8f-da8b24c82874 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:34.098917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750c558-2c52-4b1e-8005-3453c57bd68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:34.099558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750c558-2c52-4b1e-8005-3453c57bd68b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:34.099839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750c558-2c52-4b1e-8005-3453c57bd68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:34.100117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750c558-2c52-4b1e-8005-3453c57bd68b admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:34.100797 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750c558-2c52-4b1e-8005-3453c57bd68b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:34.103145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750c558-2c52-4b1e-8005-3453c57bd68b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:34.127565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1300/2594] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:34 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:04:34.131998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f38c4-5df8-4423-9a43-dc3993d53012 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:34.144263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f38c4-5df8-4423-9a43-dc3993d53012 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:34.149995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d371d6f-4463-474c-82a7-7dd1fe7b221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:34.151059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d371d6f-4463-474c-82a7-7dd1fe7b221e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:34.151490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d371d6f-4463-474c-82a7-7dd1fe7b221e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:34.151864 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d371d6f-4463-474c-82a7-7dd1fe7b221e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:34.152670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d371d6f-4463-474c-82a7-7dd1fe7b221e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:34.154381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d371d6f-4463-474c-82a7-7dd1fe7b221e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:34.167153 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1295/2595] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:34.171599 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750c558-2c52-4b1e-8005-3453c57bd68b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:34.183012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750c558-2c52-4b1e-8005-3453c57bd68b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:34.189079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3881aaa-f7a7-4c8c-9900-b790cdfbff87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:34.189946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3881aaa-f7a7-4c8c-9900-b790cdfbff87 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:34.190225 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3881aaa-f7a7-4c8c-9900-b790cdfbff87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:34.190548 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3881aaa-f7a7-4c8c-9900-b790cdfbff87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:34.191269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3881aaa-f7a7-4c8c-9900-b790cdfbff87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:34.192976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3881aaa-f7a7-4c8c-9900-b790cdfbff87 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:34.207187 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1301/2596] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:34.212338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d371d6f-4463-474c-82a7-7dd1fe7b221e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:34.226863 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d371d6f-4463-474c-82a7-7dd1fe7b221e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:34.234790 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26800fc-4146-4fd4-82c3-47db2bf0db71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:34.236026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26800fc-4146-4fd4-82c3-47db2bf0db71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:34.236747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26800fc-4146-4fd4-82c3-47db2bf0db71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:34.236747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26800fc-4146-4fd4-82c3-47db2bf0db71 admin admin] PATH_INFO: `/v3/projects/142718404c574fbe82c37ea5751157b6/users/85770558c8d847f9a55e29bdfdde8883/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:34.278119 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26800fc-4146-4fd4-82c3-47db2bf0db71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=142718404c574fbe82c37ea5751157b6, user_id=85770558c8d847f9a55e29bdfdde8883, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:34.280852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26800fc-4146-4fd4-82c3-47db2bf0db71 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:34.303287 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1296/2597] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:34 2022] PUT /identity/v3/projects/142718404c574fbe82c37ea5751157b6/users/85770558c8d847f9a55e29bdfdde8883/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:34.315819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099a57d-b8cd-4847-9fc9-1b6c5126f9d9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:34.316407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099a57d-b8cd-4847-9fc9-1b6c5126f9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:34.317093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099a57d-b8cd-4847-9fc9-1b6c5126f9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:34.351574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a099a57d-b8cd-4847-9fc9-1b6c5126f9d9 None None] MFA Rules not processed for user `85770558c8d847f9a55e29bdfdde8883`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:34.352884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a099a57d-b8cd-4847-9fc9-1b6c5126f9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:34.421539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:34.437376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1302/2598] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:34.445524 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26800fc-4146-4fd4-82c3-47db2bf0db71 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:34.447448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b26800fc-4146-4fd4-82c3-47db2bf0db71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:34.509210 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26800fc-4146-4fd4-82c3-47db2bf0db71 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:34.513100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3307040-7977-42a4-b23b-ee620d615ce5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:34.513868 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3307040-7977-42a4-b23b-ee620d615ce5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:34.514334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3307040-7977-42a4-b23b-ee620d615ce5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:34.514637 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3307040-7977-42a4-b23b-ee620d615ce5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:34.525293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3307040-7977-42a4-b23b-ee620d615ce5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:34.526837 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3307040-7977-42a4-b23b-ee620d615ce5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:34.537240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1297/2599] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:34.566385 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099a57d-b8cd-4847-9fc9-1b6c5126f9d9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:34.637034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099a57d-b8cd-4847-9fc9-1b6c5126f9d9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:34.641864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:34.642555 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:34.642839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:34.643115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] PATH_INFO: `/v3/users/85770558c8d847f9a55e29bdfdde8883` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:34.644395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=85770558c8d847f9a55e29bdfdde8883)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:34.646603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:34.699358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85770558c8d847f9a55e29bdfdde8883'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:34.708447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85770558c8d847f9a55e29bdfdde8883'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:34.709144 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85770558c8d847f9a55e29bdfdde8883'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:34.709741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85770558c8d847f9a55e29bdfdde8883'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:34.722760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85770558c8d847f9a55e29bdfdde8883'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:34.736644 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1303/2600] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:04:34 2022] DELETE /identity/v3/users/85770558c8d847f9a55e29bdfdde8883 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:34.741140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46cb41-c78f-4cb7-97b0-fea8c4db9e6c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:34.741531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46cb41-c78f-4cb7-97b0-fea8c4db9e6c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:34.741932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46cb41-c78f-4cb7-97b0-fea8c4db9e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:34.762785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-df46cb41-c78f-4cb7-97b0-fea8c4db9e6c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:34.764241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df46cb41-c78f-4cb7-97b0-fea8c4db9e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:34.768601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:34.831327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1298/2601] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:34.841424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:34.842877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:34.906191 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819bd7f-df65-449a-812d-30d23e058bea admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:34.910495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3610a2-b033-437f-a871-294a3801d107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:34.911129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3610a2-b033-437f-a871-294a3801d107 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:34.911412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3610a2-b033-437f-a871-294a3801d107 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:34.911688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3610a2-b033-437f-a871-294a3801d107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:34.922110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3610a2-b033-437f-a871-294a3801d107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:34.923699 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3610a2-b033-437f-a871-294a3801d107 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:34.934730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1304/2602] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:35.314403 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46cb41-c78f-4cb7-97b0-fea8c4db9e6c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:35.316161 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df46cb41-c78f-4cb7-97b0-fea8c4db9e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:35.328632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46cb41-c78f-4cb7-97b0-fea8c4db9e6c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:35.334336 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:35.335117 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.335462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.335807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] PATH_INFO: `/v3/projects/142718404c574fbe82c37ea5751157b6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.337057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=142718404c574fbe82c37ea5751157b6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:35.339364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:35.386025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '142718404c574fbe82c37ea5751157b6', 'request_id': 'req-947e368c-c45a-467b-a484-e00b73880f1c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:35.393776 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '142718404c574fbe82c37ea5751157b6', 'request_id': 'req-947e368c-c45a-467b-a484-e00b73880f1c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:35.399224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '142718404c574fbe82c37ea5751157b6', 'request_id': 'req-947e368c-c45a-467b-a484-e00b73880f1c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:35.409972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] The token cache is being invalidate because project 142718404c574fbe82c37ea5751157b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:04:35.410254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:35.410898 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-947e368c-c45a-467b-a484-e00b73880f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:04:35.412387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1299/2603] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:04:35 2022] DELETE /identity/v3/projects/142718404c574fbe82c37ea5751157b6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:35.417245 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99923f0e-88ce-4b82-bcce-da28bbecd56a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.417534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99923f0e-88ce-4b82-bcce-da28bbecd56a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.417812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99923f0e-88ce-4b82-bcce-da28bbecd56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.437190 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-99923f0e-88ce-4b82-bcce-da28bbecd56a None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:35.439049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-99923f0e-88ce-4b82-bcce-da28bbecd56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:35.451973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:35.503658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1305/2604] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:35.512714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a344b8-9c1b-4a86-9f33-b3bf7bf7defb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.513218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a344b8-9c1b-4a86-9f33-b3bf7bf7defb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.513329 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a344b8-9c1b-4a86-9f33-b3bf7bf7defb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.535273 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c2a344b8-9c1b-4a86-9f33-b3bf7bf7defb None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:35.536503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c2a344b8-9c1b-4a86-9f33-b3bf7bf7defb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:35.544628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1300/2605] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:35.549201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99923f0e-88ce-4b82-bcce-da28bbecd56a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:35.559104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99923f0e-88ce-4b82-bcce-da28bbecd56a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:35.564553 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545016f1-7531-468f-9cad-b42e2d309275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:35.565356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545016f1-7531-468f-9cad-b42e2d309275 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.565643 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545016f1-7531-468f-9cad-b42e2d309275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.565922 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545016f1-7531-468f-9cad-b42e2d309275 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.566930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545016f1-7531-468f-9cad-b42e2d309275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:04:35.567474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545016f1-7531-468f-9cad-b42e2d309275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:35.568927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545016f1-7531-468f-9cad-b42e2d309275 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:35.585261 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1306/2606] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:04:35.586415 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a344b8-9c1b-4a86-9f33-b3bf7bf7defb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:35.595726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a344b8-9c1b-4a86-9f33-b3bf7bf7defb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:35.600492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca59c4-b54e-453a-87a9-a371c3da0c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:35.601110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca59c4-b54e-453a-87a9-a371c3da0c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.601390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca59c4-b54e-453a-87a9-a371c3da0c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.601666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca59c4-b54e-453a-87a9-a371c3da0c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.602355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca59c4-b54e-453a-87a9-a371c3da0c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:35.604292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca59c4-b54e-453a-87a9-a371c3da0c83 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:35.635289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1301/2607] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:04:35 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:35.636569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545016f1-7531-468f-9cad-b42e2d309275 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:35.671944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545016f1-7531-468f-9cad-b42e2d309275 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:35.730693 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95b5be-df1c-4237-a913-20ac1fe27df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:35.731633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95b5be-df1c-4237-a913-20ac1fe27df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.731914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95b5be-df1c-4237-a913-20ac1fe27df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.732189 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95b5be-df1c-4237-a913-20ac1fe27df7 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.732907 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95b5be-df1c-4237-a913-20ac1fe27df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:35.734864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95b5be-df1c-4237-a913-20ac1fe27df7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:35.756811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1307/2608] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:04:35 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:04:35.761348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca59c4-b54e-453a-87a9-a371c3da0c83 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:35.772668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca59c4-b54e-453a-87a9-a371c3da0c83 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:35.777458 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce99c09-3550-4d56-8a8f-98b5d279222f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:35.778078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce99c09-3550-4d56-8a8f-98b5d279222f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.778361 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce99c09-3550-4d56-8a8f-98b5d279222f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.778676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce99c09-3550-4d56-8a8f-98b5d279222f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.779425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce99c09-3550-4d56-8a8f-98b5d279222f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:35.781047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce99c09-3550-4d56-8a8f-98b5d279222f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:35.793380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1302/2609] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:35.798044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95b5be-df1c-4237-a913-20ac1fe27df7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:35.808417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95b5be-df1c-4237-a913-20ac1fe27df7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:35.813600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc340d-9c42-4f7c-859b-ed9267831130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:35.814345 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc340d-9c42-4f7c-859b-ed9267831130 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.814655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc340d-9c42-4f7c-859b-ed9267831130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.814924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc340d-9c42-4f7c-859b-ed9267831130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.815623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc340d-9c42-4f7c-859b-ed9267831130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:35.817323 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc340d-9c42-4f7c-859b-ed9267831130 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:35.830961 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1308/2610] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:04:35.835262 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce99c09-3550-4d56-8a8f-98b5d279222f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:35.848800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce99c09-3550-4d56-8a8f-98b5d279222f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:35.854222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899e493-415d-4021-8022-fbd54f1bca94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:35.855064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899e493-415d-4021-8022-fbd54f1bca94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.855353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899e493-415d-4021-8022-fbd54f1bca94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.855633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899e493-415d-4021-8022-fbd54f1bca94 admin admin] PATH_INFO: `/v3/projects/d21f4a9d4f054748bdfb560b1995dd8e/users/b957dfaf8ccc41f1b4c8643f7bcdfa99/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.872908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899e493-415d-4021-8022-fbd54f1bca94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21f4a9d4f054748bdfb560b1995dd8e, user_id=b957dfaf8ccc41f1b4c8643f7bcdfa99, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:35.875696 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899e493-415d-4021-8022-fbd54f1bca94 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:35.890852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1303/2611] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:04:35 2022] PUT /identity/v3/projects/d21f4a9d4f054748bdfb560b1995dd8e/users/b957dfaf8ccc41f1b4c8643f7bcdfa99/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:04:35.900527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea120d1-db89-4de2-a727-425316360ded None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:35.900810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea120d1-db89-4de2-a727-425316360ded None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:35.901081 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea120d1-db89-4de2-a727-425316360ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:35.932110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-eea120d1-db89-4de2-a727-425316360ded None None] MFA Rules not processed for user `b957dfaf8ccc41f1b4c8643f7bcdfa99`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:35.933474 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eea120d1-db89-4de2-a727-425316360ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:36.007487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1309/2612] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:04:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:36.015715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899e493-415d-4021-8022-fbd54f1bca94 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:36.017594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c899e493-415d-4021-8022-fbd54f1bca94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:36.051275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:36.080750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899e493-415d-4021-8022-fbd54f1bca94 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:36.084483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b64a0-7ed8-4f06-92b2-c4938e9ddcc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:36.085127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b64a0-7ed8-4f06-92b2-c4938e9ddcc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:36.085395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b64a0-7ed8-4f06-92b2-c4938e9ddcc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:36.085672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b64a0-7ed8-4f06-92b2-c4938e9ddcc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:36.095250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b64a0-7ed8-4f06-92b2-c4938e9ddcc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:36.096750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b64a0-7ed8-4f06-92b2-c4938e9ddcc7 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:36.105982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1304/2613] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:04:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:04:36.602265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec625737-78d3-438c-8ef8-fa998e4a706d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:36.602785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec625737-78d3-438c-8ef8-fa998e4a706d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:36.602880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec625737-78d3-438c-8ef8-fa998e4a706d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:36.622977 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ec625737-78d3-438c-8ef8-fa998e4a706d None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:36.624163 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ec625737-78d3-438c-8ef8-fa998e4a706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:36.633403 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:36.681352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1310/2614] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Jul 30 05:04:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:36.693942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b64a0-7ed8-4f06-92b2-c4938e9ddcc7 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:36.695474 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c65b64a0-7ed8-4f06-92b2-c4938e9ddcc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:36.767544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b64a0-7ed8-4f06-92b2-c4938e9ddcc7 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:36.772670 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297850-9b92-41b9-8bf8-85cadc261bbc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:36.773288 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297850-9b92-41b9-8bf8-85cadc261bbc service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:36.773573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297850-9b92-41b9-8bf8-85cadc261bbc service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:36.773850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297850-9b92-41b9-8bf8-85cadc261bbc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:36.782420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08297850-9b92-41b9-8bf8-85cadc261bbc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:36.784055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08297850-9b92-41b9-8bf8-85cadc261bbc service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:36.792271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1305/2615] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:04:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 05:04:46.674765 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=19 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:46.789002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=45 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:04:55.208012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918ac0a-ffe2-41b9-a7c9-98536a70a023 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:55.208560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918ac0a-ffe2-41b9-a7c9-98536a70a023 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:55.208560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918ac0a-ffe2-41b9-a7c9-98536a70a023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:55.228450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4918ac0a-ffe2-41b9-a7c9-98536a70a023 None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:04:55.229684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4918ac0a-ffe2-41b9-a7c9-98536a70a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:04:55.237177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1311/2616] 10.222.0.20 () {58 vars in 967 bytes} [Sat Jul 30 05:04:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:04:55.251745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297850-9b92-41b9-8bf8-85cadc261bbc service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:04:55.264253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297850-9b92-41b9-8bf8-85cadc261bbc service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:04:55.270160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9aadf8-7c6d-4cbc-95d9-a9379361529b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:04:55.270800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9aadf8-7c6d-4cbc-95d9-a9379361529b service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:04:55.271084 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9aadf8-7c6d-4cbc-95d9-a9379361529b service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:04:55.271366 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9aadf8-7c6d-4cbc-95d9-a9379361529b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:04:55.280192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9aadf8-7c6d-4cbc-95d9-a9379361529b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:04:55.282724 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9aadf8-7c6d-4cbc-95d9-a9379361529b service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:04:55.291189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1306/2617] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:04:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 05:05:05.222981 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:05.287743 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:10.121763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918ac0a-ffe2-41b9-a7c9-98536a70a023 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:10.123192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4918ac0a-ffe2-41b9-a7c9-98536a70a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:10.178870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918ac0a-ffe2-41b9-a7c9-98536a70a023 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:10.182495 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e44701-a2ab-4583-a532-e47b70ede8c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:10.183111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e44701-a2ab-4583-a532-e47b70ede8c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:10.183390 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e44701-a2ab-4583-a532-e47b70ede8c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:10.183663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e44701-a2ab-4583-a532-e47b70ede8c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:10.185412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d2e44701-a2ab-4583-a532-e47b70ede8c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:10.247156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e44701-a2ab-4583-a532-e47b70ede8c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:10.248598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e44701-a2ab-4583-a532-e47b70ede8c1 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:10.258475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1312/2618] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:11.642412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9aadf8-7c6d-4cbc-95d9-a9379361529b service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:11.652960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9aadf8-7c6d-4cbc-95d9-a9379361529b service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:11.657941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:11.658553 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:11.658901 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:11.659185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] PATH_INFO: `/v3/users/b957dfaf8ccc41f1b4c8643f7bcdfa99` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:11.660493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b957dfaf8ccc41f1b4c8643f7bcdfa99)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:11.662428 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:11.709875 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b957dfaf8ccc41f1b4c8643f7bcdfa99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:11.717000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b957dfaf8ccc41f1b4c8643f7bcdfa99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:11.717674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b957dfaf8ccc41f1b4c8643f7bcdfa99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:11.718293 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b957dfaf8ccc41f1b4c8643f7bcdfa99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:11.732319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b957dfaf8ccc41f1b4c8643f7bcdfa99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:11.747588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1307/2619] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:11 2022] DELETE /identity/v3/users/b957dfaf8ccc41f1b4c8643f7bcdfa99 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:11.752579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ef38-18c6-498b-a912-66826e87a8d1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:11.753240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ef38-18c6-498b-a912-66826e87a8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:11.753537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ef38-18c6-498b-a912-66826e87a8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:11.773172 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7621ef38-18c6-498b-a912-66826e87a8d1 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:11.774487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7621ef38-18c6-498b-a912-66826e87a8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:11.837439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1313/2620] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:11.849922 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:11.851679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:11.935470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c1a3b-b854-4e8d-ba96-df021478db9a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:11.939829 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7e0a3-1e3f-42d8-822d-865e3b5a22f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:11.940465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7e0a3-1e3f-42d8-822d-865e3b5a22f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:11.940745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7e0a3-1e3f-42d8-822d-865e3b5a22f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:11.941017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7e0a3-1e3f-42d8-822d-865e3b5a22f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:11.950787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7e0a3-1e3f-42d8-822d-865e3b5a22f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:11.952238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7e0a3-1e3f-42d8-822d-865e3b5a22f2 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:11.963160 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1308/2621] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:12.271586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621ef38-18c6-498b-a912-66826e87a8d1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:12.273274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7621ef38-18c6-498b-a912-66826e87a8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:12.288884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621ef38-18c6-498b-a912-66826e87a8d1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:12.295494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:12.296820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:12.297415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:12.298776 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] PATH_INFO: `/v3/projects/d21f4a9d4f054748bdfb560b1995dd8e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:12.301080 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d21f4a9d4f054748bdfb560b1995dd8e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:12.305021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:12.364964 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:12.368022 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd21f4a9d4f054748bdfb560b1995dd8e', 'request_id': 'req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:12.378912 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd21f4a9d4f054748bdfb560b1995dd8e', 'request_id': 'req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:12.385277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd21f4a9d4f054748bdfb560b1995dd8e', 'request_id': 'req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:12.398537 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] The token cache is being invalidate because project d21f4a9d4f054748bdfb560b1995dd8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:12.399075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:12.400149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:12.401979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1314/2622] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:12 2022] DELETE /identity/v3/projects/d21f4a9d4f054748bdfb560b1995dd8e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:13.490052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7e0a3-1e3f-42d8-822d-865e3b5a22f2 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:13.491524 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1e7e0a3-1e3f-42d8-822d-865e3b5a22f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:13.553989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:13.555451 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:13.574227 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7e0a3-1e3f-42d8-822d-865e3b5a22f2 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:13.587360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:13.626386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61b0b1-e8c6-4752-a9b9-4e06581323c0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:13.631045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:13.631592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:13.631871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:13.632147 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] PATH_INFO: `/v3/users/a8abb58f9c274e568d2d0fe3d4df11de` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:13.633358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8abb58f9c274e568d2d0fe3d4df11de)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:13.635317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:13.645813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc840e30-1ff3-453c-83bb-a0c6d9d50e27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:13.646483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc840e30-1ff3-453c-83bb-a0c6d9d50e27 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:13.646767 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc840e30-1ff3-453c-83bb-a0c6d9d50e27 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:13.647042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc840e30-1ff3-453c-83bb-a0c6d9d50e27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:13.648830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc840e30-1ff3-453c-83bb-a0c6d9d50e27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:13.660708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc840e30-1ff3-453c-83bb-a0c6d9d50e27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:13.662392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc840e30-1ff3-453c-83bb-a0c6d9d50e27 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:13.673089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1309/2623] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:13.683993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8abb58f9c274e568d2d0fe3d4df11de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:13.692860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8abb58f9c274e568d2d0fe3d4df11de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:13.693560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8abb58f9c274e568d2d0fe3d4df11de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:13.694174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8abb58f9c274e568d2d0fe3d4df11de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:13.707478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-082f33ee-8d69-4af5-8556-fb20ffad40a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8abb58f9c274e568d2d0fe3d4df11de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:13.720503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1315/2624] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:13 2022] DELETE /identity/v3/users/a8abb58f9c274e568d2d0fe3d4df11de => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:05:13.895796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc840e30-1ff3-453c-83bb-a0c6d9d50e27 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:13.908447 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc840e30-1ff3-453c-83bb-a0c6d9d50e27 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:13.965834 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:13.966400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:13.966704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:13.966973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] PATH_INFO: `/v3/projects/e01500cb03c74d1082090cd1c1836169` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:13.968112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01500cb03c74d1082090cd1c1836169)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:13.970747 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:14.014562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01500cb03c74d1082090cd1c1836169', 'request_id': 'req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:14.023876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01500cb03c74d1082090cd1c1836169', 'request_id': 'req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:14.029777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01500cb03c74d1082090cd1c1836169', 'request_id': 'req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:14.041878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] The token cache is being invalidate because project e01500cb03c74d1082090cd1c1836169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:14.042167 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:14.042804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8ab0b7b0-4dec-4eb3-9b44-67f779c27701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:14.044314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1310/2625] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:13 2022] DELETE /identity/v3/projects/e01500cb03c74d1082090cd1c1836169 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:14.049610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c82e3e-67b0-49e6-b494-b1796c57fa9d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.049902 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c82e3e-67b0-49e6-b494-b1796c57fa9d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.050174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c82e3e-67b0-49e6-b494-b1796c57fa9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.070284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-84c82e3e-67b0-49e6-b494-b1796c57fa9d None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:14.071513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-84c82e3e-67b0-49e6-b494-b1796c57fa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:14.119165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:14.132623 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1316/2626] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:14.140404 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff82b1-a379-4e10-9aa4-8bfe6c221d28 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.140690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff82b1-a379-4e10-9aa4-8bfe6c221d28 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.140964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff82b1-a379-4e10-9aa4-8bfe6c221d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.161478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-caff82b1-a379-4e10-9aa4-8bfe6c221d28 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:14.162714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-caff82b1-a379-4e10-9aa4-8bfe6c221d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:14.171057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1311/2627] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:14.175439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c82e3e-67b0-49e6-b494-b1796c57fa9d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:14.185083 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c82e3e-67b0-49e6-b494-b1796c57fa9d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:14.190271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2de676-76e7-4b9e-b7ec-85f4d3ea49be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:14.190903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2de676-76e7-4b9e-b7ec-85f4d3ea49be admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.191178 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2de676-76e7-4b9e-b7ec-85f4d3ea49be admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.191445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2de676-76e7-4b9e-b7ec-85f4d3ea49be admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.192131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2de676-76e7-4b9e-b7ec-85f4d3ea49be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:05:14.192427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2de676-76e7-4b9e-b7ec-85f4d3ea49be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:14.194005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2de676-76e7-4b9e-b7ec-85f4d3ea49be admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:14.206142 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1317/2628] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:05:14.210382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff82b1-a379-4e10-9aa4-8bfe6c221d28 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:14.219732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff82b1-a379-4e10-9aa4-8bfe6c221d28 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:14.224341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671e5b8-73be-47d1-9507-8ab3eb41395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:14.224947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671e5b8-73be-47d1-9507-8ab3eb41395c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.225226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671e5b8-73be-47d1-9507-8ab3eb41395c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.225499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671e5b8-73be-47d1-9507-8ab3eb41395c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.226159 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671e5b8-73be-47d1-9507-8ab3eb41395c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:14.228095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671e5b8-73be-47d1-9507-8ab3eb41395c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:14.255561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1312/2629] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:05:14 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:14.259753 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2de676-76e7-4b9e-b7ec-85f4d3ea49be admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:14.270295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2de676-76e7-4b9e-b7ec-85f4d3ea49be admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:14.326580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1184bab-aa11-4ec9-88b5-271bdc8737a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:14.327193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1184bab-aa11-4ec9-88b5-271bdc8737a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.327467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1184bab-aa11-4ec9-88b5-271bdc8737a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.327736 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1184bab-aa11-4ec9-88b5-271bdc8737a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.328380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1184bab-aa11-4ec9-88b5-271bdc8737a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:14.331133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1184bab-aa11-4ec9-88b5-271bdc8737a7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:14.352223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1318/2630] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:05:14 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:05:14.356364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671e5b8-73be-47d1-9507-8ab3eb41395c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:14.367618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671e5b8-73be-47d1-9507-8ab3eb41395c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:14.372238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78272717-7a3f-47aa-b1ef-72a382eeb7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:14.372845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78272717-7a3f-47aa-b1ef-72a382eeb7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.373123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78272717-7a3f-47aa-b1ef-72a382eeb7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.373397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78272717-7a3f-47aa-b1ef-72a382eeb7e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.374083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78272717-7a3f-47aa-b1ef-72a382eeb7e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:14.376261 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78272717-7a3f-47aa-b1ef-72a382eeb7e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:14.378341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:14.387463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1313/2631] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:14.392216 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1184bab-aa11-4ec9-88b5-271bdc8737a7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:14.402194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1184bab-aa11-4ec9-88b5-271bdc8737a7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:14.406810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f94d58-611c-42e5-9b1a-88dfa2a7c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:14.407630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f94d58-611c-42e5-9b1a-88dfa2a7c0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.407909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f94d58-611c-42e5-9b1a-88dfa2a7c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.408177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f94d58-611c-42e5-9b1a-88dfa2a7c0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.408861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f94d58-611c-42e5-9b1a-88dfa2a7c0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:14.410591 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f94d58-611c-42e5-9b1a-88dfa2a7c0ce admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:14.421254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1319/2632] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:14.425378 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78272717-7a3f-47aa-b1ef-72a382eeb7e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:14.435302 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78272717-7a3f-47aa-b1ef-72a382eeb7e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:14.440525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69924c7-b8e3-49d5-853e-a7b7384cdd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:14.441171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69924c7-b8e3-49d5-853e-a7b7384cdd72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.441453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69924c7-b8e3-49d5-853e-a7b7384cdd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.441730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69924c7-b8e3-49d5-853e-a7b7384cdd72 admin admin] PATH_INFO: `/v3/projects/696413e60ae44275acd7a718e7be1315/users/12064bfdaa314eb5a0542b15eb5cda75/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.456889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69924c7-b8e3-49d5-853e-a7b7384cdd72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696413e60ae44275acd7a718e7be1315, user_id=12064bfdaa314eb5a0542b15eb5cda75, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:14.459590 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69924c7-b8e3-49d5-853e-a7b7384cdd72 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:14.472662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1314/2633] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:14 2022] PUT /identity/v3/projects/696413e60ae44275acd7a718e7be1315/users/12064bfdaa314eb5a0542b15eb5cda75/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:14.482564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3597cc-3c26-4f26-8fbd-fe6c55bded3f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.482846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3597cc-3c26-4f26-8fbd-fe6c55bded3f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.483162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3597cc-3c26-4f26-8fbd-fe6c55bded3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.511477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1c3597cc-3c26-4f26-8fbd-fe6c55bded3f None None] MFA Rules not processed for user `12064bfdaa314eb5a0542b15eb5cda75`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:14.512713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1c3597cc-3c26-4f26-8fbd-fe6c55bded3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:14.582011 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1320/2634] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:14.589488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69924c7-b8e3-49d5-853e-a7b7384cdd72 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:14.591322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a69924c7-b8e3-49d5-853e-a7b7384cdd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:14.652136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69924c7-b8e3-49d5-853e-a7b7384cdd72 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:14.656102 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48dd91-eb3f-44d6-b51e-eb1e70b2db72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:14.656726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48dd91-eb3f-44d6-b51e-eb1e70b2db72 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:14.657132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48dd91-eb3f-44d6-b51e-eb1e70b2db72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:14.657825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48dd91-eb3f-44d6-b51e-eb1e70b2db72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:14.666308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48dd91-eb3f-44d6-b51e-eb1e70b2db72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:14.667796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48dd91-eb3f-44d6-b51e-eb1e70b2db72 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:14.677594 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1315/2635] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:14 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:19.885562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3597cc-3c26-4f26-8fbd-fe6c55bded3f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:19.887208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1c3597cc-3c26-4f26-8fbd-fe6c55bded3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:19.944910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:19.949101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3597cc-3c26-4f26-8fbd-fe6c55bded3f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:19.953723 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e7279-70c6-457b-96cc-2653ccdc6db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:19.954903 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e7279-70c6-457b-96cc-2653ccdc6db1 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:19.955176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e7279-70c6-457b-96cc-2653ccdc6db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:19.955527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e7279-70c6-457b-96cc-2653ccdc6db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:19.957298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-060e7279-70c6-457b-96cc-2653ccdc6db1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:20.015771 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e7279-70c6-457b-96cc-2653ccdc6db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:20.017893 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e7279-70c6-457b-96cc-2653ccdc6db1 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:20.027445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1321/2636] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:23.025962 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48dd91-eb3f-44d6-b51e-eb1e70b2db72 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:23.092480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48dd91-eb3f-44d6-b51e-eb1e70b2db72 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:23.097181 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:23.097871 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:23.098155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:23.098429 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] PATH_INFO: `/v3/users/12064bfdaa314eb5a0542b15eb5cda75` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:23.099632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12064bfdaa314eb5a0542b15eb5cda75)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:23.101559 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:23.151222 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12064bfdaa314eb5a0542b15eb5cda75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.159089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12064bfdaa314eb5a0542b15eb5cda75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.159777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12064bfdaa314eb5a0542b15eb5cda75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.160381 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12064bfdaa314eb5a0542b15eb5cda75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.172698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12064bfdaa314eb5a0542b15eb5cda75'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.176731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:23.189092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1316/2637] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:23 2022] DELETE /identity/v3/users/12064bfdaa314eb5a0542b15eb5cda75 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:23.193556 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c3220-7108-4650-8837-af98bf5a6241 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:23.194060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c3220-7108-4650-8837-af98bf5a6241 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:23.194162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c3220-7108-4650-8837-af98bf5a6241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:23.214347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-426c3220-7108-4650-8837-af98bf5a6241 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:23.215596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-426c3220-7108-4650-8837-af98bf5a6241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:23.274619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1322/2638] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:05:23.285824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:23.287633 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:23.350932 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e149d-3e8a-4184-b1cf-72b963e16d42 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:23.354680 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e0bbd-f3bd-45dc-bffb-4116f5e2eb68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:23.355346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e0bbd-f3bd-45dc-bffb-4116f5e2eb68 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:23.355626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e0bbd-f3bd-45dc-bffb-4116f5e2eb68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:23.355939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e0bbd-f3bd-45dc-bffb-4116f5e2eb68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:23.365616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e0bbd-f3bd-45dc-bffb-4116f5e2eb68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:23.367148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e0bbd-f3bd-45dc-bffb-4116f5e2eb68 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:23.378342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1317/2639] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:05:23.684784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c3220-7108-4650-8837-af98bf5a6241 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:23.686782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-426c3220-7108-4650-8837-af98bf5a6241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:23.702953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c3220-7108-4650-8837-af98bf5a6241 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:23.709141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:23.709995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:23.710121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:23.710605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] PATH_INFO: `/v3/projects/696413e60ae44275acd7a718e7be1315` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:23.711962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=696413e60ae44275acd7a718e7be1315)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:23.713722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:23.763946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696413e60ae44275acd7a718e7be1315', 'request_id': 'req-e7102119-4c5a-4d0a-879a-96072f011aa9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.771735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696413e60ae44275acd7a718e7be1315', 'request_id': 'req-e7102119-4c5a-4d0a-879a-96072f011aa9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.777290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696413e60ae44275acd7a718e7be1315', 'request_id': 'req-e7102119-4c5a-4d0a-879a-96072f011aa9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.788852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] The token cache is being invalidate because project 696413e60ae44275acd7a718e7be1315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:23.789133 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.789750 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e7102119-4c5a-4d0a-879a-96072f011aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:23.791282 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1323/2640] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:23 2022] DELETE /identity/v3/projects/696413e60ae44275acd7a718e7be1315 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:23.796391 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502eb52-94ac-41de-87f6-25391f77d6d5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:23.796673 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502eb52-94ac-41de-87f6-25391f77d6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:23.796947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502eb52-94ac-41de-87f6-25391f77d6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:23.816715 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7502eb52-94ac-41de-87f6-25391f77d6d5 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:23.817939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7502eb52-94ac-41de-87f6-25391f77d6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:23.886002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1318/2641] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:05:23.895352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7de43-c73f-4f14-ba9d-e5911391b638 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:23.895667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7de43-c73f-4f14-ba9d-e5911391b638 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:23.895919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7de43-c73f-4f14-ba9d-e5911391b638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:23.918415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4cf7de43-c73f-4f14-ba9d-e5911391b638 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:23.920076 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4cf7de43-c73f-4f14-ba9d-e5911391b638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:23.928339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1324/2642] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:05:23.932770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502eb52-94ac-41de-87f6-25391f77d6d5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:23.943661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502eb52-94ac-41de-87f6-25391f77d6d5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:23.948382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6db0c-ecd2-4164-a369-b898b99fcedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:23.949003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6db0c-ecd2-4164-a369-b898b99fcedc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:23.949280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6db0c-ecd2-4164-a369-b898b99fcedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:23.949549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6db0c-ecd2-4164-a369-b898b99fcedc admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:23.950240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6db0c-ecd2-4164-a369-b898b99fcedc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:05:23.950566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6db0c-ecd2-4164-a369-b898b99fcedc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:23.952165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6db0c-ecd2-4164-a369-b898b99fcedc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:23.965821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1319/2643] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:05:23.970249 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7de43-c73f-4f14-ba9d-e5911391b638 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:23.980492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7de43-c73f-4f14-ba9d-e5911391b638 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:23.985754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730f2d4-1c64-4e02-9986-1071089fe87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:23.986516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730f2d4-1c64-4e02-9986-1071089fe87b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:23.986861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730f2d4-1c64-4e02-9986-1071089fe87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:23.987201 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730f2d4-1c64-4e02-9986-1071089fe87b admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:23.988450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730f2d4-1c64-4e02-9986-1071089fe87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:23.990713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730f2d4-1c64-4e02-9986-1071089fe87b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.025389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:24.042620 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6db0c-ecd2-4164-a369-b898b99fcedc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.044735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1325/2644] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:05:23 2022] POST /identity/v3/projects => generated 365 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:24.062744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6db0c-ecd2-4164-a369-b898b99fcedc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.141694 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a5f3d-ab79-4563-bba0-f160dfb4e479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.142817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a5f3d-ab79-4563-bba0-f160dfb4e479 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.143086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a5f3d-ab79-4563-bba0-f160dfb4e479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.143360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a5f3d-ab79-4563-bba0-f160dfb4e479 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.144014 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a5f3d-ab79-4563-bba0-f160dfb4e479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.145941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a5f3d-ab79-4563-bba0-f160dfb4e479 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.166794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1320/2645] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:05:24 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:24.171329 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730f2d4-1c64-4e02-9986-1071089fe87b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.182547 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730f2d4-1c64-4e02-9986-1071089fe87b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.187415 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee657418-ecca-4f48-a370-94d67ac473c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.188079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee657418-ecca-4f48-a370-94d67ac473c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.188412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee657418-ecca-4f48-a370-94d67ac473c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.189811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee657418-ecca-4f48-a370-94d67ac473c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.190648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee657418-ecca-4f48-a370-94d67ac473c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.192318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee657418-ecca-4f48-a370-94d67ac473c0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.202779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1326/2646] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:05:24.207578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a5f3d-ab79-4563-bba0-f160dfb4e479 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.218579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a5f3d-ab79-4563-bba0-f160dfb4e479 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.223742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad76a8-bef9-4128-a14d-e684798b1e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.224349 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad76a8-bef9-4128-a14d-e684798b1e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.224624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad76a8-bef9-4128-a14d-e684798b1e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.224894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad76a8-bef9-4128-a14d-e684798b1e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.225587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad76a8-bef9-4128-a14d-e684798b1e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.227211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad76a8-bef9-4128-a14d-e684798b1e47 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.238047 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1321/2647] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:24.242580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee657418-ecca-4f48-a370-94d67ac473c0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.253535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee657418-ecca-4f48-a370-94d67ac473c0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.258354 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d762db3-5c01-4379-a592-1f8d002ddca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.259068 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d762db3-5c01-4379-a592-1f8d002ddca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.259349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d762db3-5c01-4379-a592-1f8d002ddca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.259625 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d762db3-5c01-4379-a592-1f8d002ddca1 admin admin] PATH_INFO: `/v3/projects/a33da2c9e16d4470b30b7566fb47db6c/users/c8f182e9fffc4e349ca057415c6547ef/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.275169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d762db3-5c01-4379-a592-1f8d002ddca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33da2c9e16d4470b30b7566fb47db6c, user_id=c8f182e9fffc4e349ca057415c6547ef, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.277858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d762db3-5c01-4379-a592-1f8d002ddca1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.290648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1327/2648] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:24 2022] PUT /identity/v3/projects/a33da2c9e16d4470b30b7566fb47db6c/users/c8f182e9fffc4e349ca057415c6547ef/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:24.303064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e42fa-411c-4f2b-93ab-bcbd983133aa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.303357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e42fa-411c-4f2b-93ab-bcbd983133aa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.303851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e42fa-411c-4f2b-93ab-bcbd983133aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.311984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:24.333174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8c0e42fa-411c-4f2b-93ab-bcbd983133aa None None] MFA Rules not processed for user `c8f182e9fffc4e349ca057415c6547ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:24.334390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c0e42fa-411c-4f2b-93ab-bcbd983133aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:24.402546 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1322/2649] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:24.407027 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d762db3-5c01-4379-a592-1f8d002ddca1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.470962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d762db3-5c01-4379-a592-1f8d002ddca1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.475821 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d7754-fc56-4d29-82a8-e3bb1a27655f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.476438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d7754-fc56-4d29-82a8-e3bb1a27655f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.476717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d7754-fc56-4d29-82a8-e3bb1a27655f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.477015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d7754-fc56-4d29-82a8-e3bb1a27655f admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.477680 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d7754-fc56-4d29-82a8-e3bb1a27655f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.479600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d7754-fc56-4d29-82a8-e3bb1a27655f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.505322 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1328/2650] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:05:24 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:24.509565 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e42fa-411c-4f2b-93ab-bcbd983133aa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.520923 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e42fa-411c-4f2b-93ab-bcbd983133aa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.578097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286a0ca-c812-4871-b215-21ea74000dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.578755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286a0ca-c812-4871-b215-21ea74000dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.579037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286a0ca-c812-4871-b215-21ea74000dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.579300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286a0ca-c812-4871-b215-21ea74000dd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.580496 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286a0ca-c812-4871-b215-21ea74000dd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.582416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286a0ca-c812-4871-b215-21ea74000dd4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.606484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1323/2651] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:05:24 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:24.609135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d7754-fc56-4d29-82a8-e3bb1a27655f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.620165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d7754-fc56-4d29-82a8-e3bb1a27655f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.625538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca35683-4d17-42c4-a339-24cb781bfe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.626151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca35683-4d17-42c4-a339-24cb781bfe69 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.626454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca35683-4d17-42c4-a339-24cb781bfe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.626761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca35683-4d17-42c4-a339-24cb781bfe69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.627810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca35683-4d17-42c4-a339-24cb781bfe69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.629402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca35683-4d17-42c4-a339-24cb781bfe69 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.645545 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1329/2652] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:24.649625 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286a0ca-c812-4871-b215-21ea74000dd4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.661240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286a0ca-c812-4871-b215-21ea74000dd4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.665924 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21de81-3e30-4e42-8ca2-0e3fef16d42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.666639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21de81-3e30-4e42-8ca2-0e3fef16d42a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.666917 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21de81-3e30-4e42-8ca2-0e3fef16d42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.667220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21de81-3e30-4e42-8ca2-0e3fef16d42a admin admin] PATH_INFO: `/v3/projects/aa82a84a61a7446fb9d323ddb4a6cfef/users/bad28a35a9294fe299ae8484fce43133/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.685548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21de81-3e30-4e42-8ca2-0e3fef16d42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa82a84a61a7446fb9d323ddb4a6cfef, user_id=bad28a35a9294fe299ae8484fce43133, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.688407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21de81-3e30-4e42-8ca2-0e3fef16d42a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.704536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1324/2653] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:24 2022] PUT /identity/v3/projects/aa82a84a61a7446fb9d323ddb4a6cfef/users/bad28a35a9294fe299ae8484fce43133/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:05:24.710075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca35683-4d17-42c4-a339-24cb781bfe69 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.720810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca35683-4d17-42c4-a339-24cb781bfe69 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.779991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b16cb0-4431-440f-9f5a-49b62c4f7730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.780606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b16cb0-4431-440f-9f5a-49b62c4f7730 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.780884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b16cb0-4431-440f-9f5a-49b62c4f7730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.781154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b16cb0-4431-440f-9f5a-49b62c4f7730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.781844 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b16cb0-4431-440f-9f5a-49b62c4f7730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.783641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b16cb0-4431-440f-9f5a-49b62c4f7730 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.794836 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1330/2654] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:24.799283 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21de81-3e30-4e42-8ca2-0e3fef16d42a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.810325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21de81-3e30-4e42-8ca2-0e3fef16d42a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:24.815788 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bebca-22c4-4905-91a8-d76632c7c5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:24.816450 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bebca-22c4-4905-91a8-d76632c7c5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.816727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bebca-22c4-4905-91a8-d76632c7c5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.817004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bebca-22c4-4905-91a8-d76632c7c5f7 admin admin] PATH_INFO: `/v3/projects/aa82a84a61a7446fb9d323ddb4a6cfef/users/bad28a35a9294fe299ae8484fce43133/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.818972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bebca-22c4-4905-91a8-d76632c7c5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa82a84a61a7446fb9d323ddb4a6cfef, user_id=bad28a35a9294fe299ae8484fce43133, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:24.821587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bebca-22c4-4905-91a8-d76632c7c5f7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:24.832646 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1325/2655] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:24 2022] PUT /identity/v3/projects/aa82a84a61a7446fb9d323ddb4a6cfef/users/bad28a35a9294fe299ae8484fce43133/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:05:24.844332 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d92c0-57be-404a-80d5-0c7d0d89aab8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:24.844621 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d92c0-57be-404a-80d5-0c7d0d89aab8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:24.844899 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d92c0-57be-404a-80d5-0c7d0d89aab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:24.875186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9d1d92c0-57be-404a-80d5-0c7d0d89aab8 None None] MFA Rules not processed for user `bad28a35a9294fe299ae8484fce43133`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:24.876401 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9d1d92c0-57be-404a-80d5-0c7d0d89aab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:24.896007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:24.946292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1331/2656] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:24.957709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bebca-22c4-4905-91a8-d76632c7c5f7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:24.959451 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c18bebca-22c4-4905-91a8-d76632c7c5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:25.011882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:25.018535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bebca-22c4-4905-91a8-d76632c7c5f7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:25.025224 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6bc4-8fdc-4abd-92bb-3ec0f06d933b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:25.025844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6bc4-8fdc-4abd-92bb-3ec0f06d933b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:25.026123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6bc4-8fdc-4abd-92bb-3ec0f06d933b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:25.026400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6bc4-8fdc-4abd-92bb-3ec0f06d933b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:25.036325 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca6bc4-8fdc-4abd-92bb-3ec0f06d933b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:25.037757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca6bc4-8fdc-4abd-92bb-3ec0f06d933b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:25.047520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1326/2657] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:25.121526 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d92c0-57be-404a-80d5-0c7d0d89aab8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:25.131325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d92c0-57be-404a-80d5-0c7d0d89aab8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:25.134957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13ae8f-62a2-458e-a0f5-5321e4d2f51f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:25.135563 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13ae8f-62a2-458e-a0f5-5321e4d2f51f service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:25.135842 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13ae8f-62a2-458e-a0f5-5321e4d2f51f service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:25.136114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13ae8f-62a2-458e-a0f5-5321e4d2f51f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:25.192904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13ae8f-62a2-458e-a0f5-5321e4d2f51f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:25.194334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13ae8f-62a2-458e-a0f5-5321e4d2f51f service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:25.203211 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1332/2658] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:25.956069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6bc4-8fdc-4abd-92bb-3ec0f06d933b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:25.957529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5ca6bc4-8fdc-4abd-92bb-3ec0f06d933b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:26.015595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6bc4-8fdc-4abd-92bb-3ec0f06d933b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:26.020307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363588c0-5159-482c-875d-c1e27e24c831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:26.021038 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363588c0-5159-482c-875d-c1e27e24c831 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:26.021327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363588c0-5159-482c-875d-c1e27e24c831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:26.021624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363588c0-5159-482c-875d-c1e27e24c831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:26.023505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-363588c0-5159-482c-875d-c1e27e24c831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:26.087763 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363588c0-5159-482c-875d-c1e27e24c831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:26.089132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363588c0-5159-482c-875d-c1e27e24c831 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:26.098972 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1327/2659] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:27.966745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13ae8f-62a2-458e-a0f5-5321e4d2f51f service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:27.978400 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13ae8f-62a2-458e-a0f5-5321e4d2f51f service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:27.984562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:27.985651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:27.986158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:27.986570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] PATH_INFO: `/v3/users/c8f182e9fffc4e349ca057415c6547ef` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:27.988269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f182e9fffc4e349ca057415c6547ef)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:27.990542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:28.051032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f182e9fffc4e349ca057415c6547ef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.060677 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f182e9fffc4e349ca057415c6547ef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.061521 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f182e9fffc4e349ca057415c6547ef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.062357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f182e9fffc4e349ca057415c6547ef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.076870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f182e9fffc4e349ca057415c6547ef'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.093957 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1333/2660] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:27 2022] DELETE /identity/v3/users/c8f182e9fffc4e349ca057415c6547ef => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:28.099241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2382696-5e4e-4be3-abf0-47b9632a2cb8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:28.100335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2382696-5e4e-4be3-abf0-47b9632a2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:28.100800 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2382696-5e4e-4be3-abf0-47b9632a2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:28.124723 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e2382696-5e4e-4be3-abf0-47b9632a2cb8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:28.126277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2382696-5e4e-4be3-abf0-47b9632a2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:28.196379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1328/2661] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:28.207484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:28.209123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:28.253102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:28.295554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cdb28-7c77-42fc-a64a-54b9e62c25d3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:28.300716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bb7cc-b6da-4595-be16-9ddbac6c0bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:28.301334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bb7cc-b6da-4595-be16-9ddbac6c0bff service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:28.301609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bb7cc-b6da-4595-be16-9ddbac6c0bff service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:28.301879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bb7cc-b6da-4595-be16-9ddbac6c0bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:28.312078 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bb7cc-b6da-4595-be16-9ddbac6c0bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:28.314267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bb7cc-b6da-4595-be16-9ddbac6c0bff service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:28.325445 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1334/2662] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:28.603267 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2382696-5e4e-4be3-abf0-47b9632a2cb8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:28.605194 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2382696-5e4e-4be3-abf0-47b9632a2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:28.624493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2382696-5e4e-4be3-abf0-47b9632a2cb8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:28.630832 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:28.631754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:28.631973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:28.632248 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] PATH_INFO: `/v3/projects/a33da2c9e16d4470b30b7566fb47db6c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:28.633432 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33da2c9e16d4470b30b7566fb47db6c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:28.636401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:28.680885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:28.693254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33da2c9e16d4470b30b7566fb47db6c', 'request_id': 'req-41909eec-16ae-479c-a28d-05dca35da8b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.705390 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33da2c9e16d4470b30b7566fb47db6c', 'request_id': 'req-41909eec-16ae-479c-a28d-05dca35da8b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.712977 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33da2c9e16d4470b30b7566fb47db6c', 'request_id': 'req-41909eec-16ae-479c-a28d-05dca35da8b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.727824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] The token cache is being invalidate because project a33da2c9e16d4470b30b7566fb47db6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:28.728298 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.729027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.731269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1329/2663] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:28 2022] DELETE /identity/v3/projects/a33da2c9e16d4470b30b7566fb47db6c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:28.735381 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bb7cc-b6da-4595-be16-9ddbac6c0bff service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:28.746049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bb7cc-b6da-4595-be16-9ddbac6c0bff service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:28.821757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:28.822407 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:28.822720 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:28.822993 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] PATH_INFO: `/v3/users/bad28a35a9294fe299ae8484fce43133` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:28.824192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad28a35a9294fe299ae8484fce43133)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:28.826106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:28.881423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad28a35a9294fe299ae8484fce43133'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.890598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad28a35a9294fe299ae8484fce43133'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.891542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad28a35a9294fe299ae8484fce43133'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.893598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad28a35a9294fe299ae8484fce43133'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.911692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-43049b73-9b16-456f-94d7-9395a87f5a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad28a35a9294fe299ae8484fce43133'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:28.928466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1335/2664] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:28 2022] DELETE /identity/v3/users/bad28a35a9294fe299ae8484fce43133 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:29.402408 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:29.403909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:29.430185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41909eec-16ae-479c-a28d-05dca35da8b7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:29.532274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:29.533487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:29.533769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:29.534064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] PATH_INFO: `/v3/projects/aa82a84a61a7446fb9d323ddb4a6cfef` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:29.547065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa82a84a61a7446fb9d323ddb4a6cfef)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:29.552291 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:29.603439 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa82a84a61a7446fb9d323ddb4a6cfef', 'request_id': 'req-860f98fa-fe41-44d7-8394-43d5d4912a81'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:29.616275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa82a84a61a7446fb9d323ddb4a6cfef', 'request_id': 'req-860f98fa-fe41-44d7-8394-43d5d4912a81'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:29.622034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa82a84a61a7446fb9d323ddb4a6cfef', 'request_id': 'req-860f98fa-fe41-44d7-8394-43d5d4912a81'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:29.634315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] The token cache is being invalidate because project aa82a84a61a7446fb9d323ddb4a6cfef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:29.634754 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:29.635339 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-860f98fa-fe41-44d7-8394-43d5d4912a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:29.636949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1330/2665] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:29 2022] DELETE /identity/v3/projects/aa82a84a61a7446fb9d323ddb4a6cfef => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:29.642018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234c5d1-a065-4bfb-b75c-58da7cf0bf81 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:29.642535 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234c5d1-a065-4bfb-b75c-58da7cf0bf81 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:29.642785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234c5d1-a065-4bfb-b75c-58da7cf0bf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:29.663435 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e234c5d1-a065-4bfb-b75c-58da7cf0bf81 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:29.664649 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e234c5d1-a065-4bfb-b75c-58da7cf0bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:29.711706 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:29.728579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1336/2666] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:05:29.736985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcfebf-107e-49a3-ad10-255cd2e0613b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:29.737259 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcfebf-107e-49a3-ad10-255cd2e0613b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:29.737527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcfebf-107e-49a3-ad10-255cd2e0613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:29.758226 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7bdcfebf-107e-49a3-ad10-255cd2e0613b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:29.759494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7bdcfebf-107e-49a3-ad10-255cd2e0613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:29.767630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1331/2667] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:29.771863 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234c5d1-a065-4bfb-b75c-58da7cf0bf81 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:29.781690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234c5d1-a065-4bfb-b75c-58da7cf0bf81 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:29.787195 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00c860-a034-406c-b502-fd026772c38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:29.787747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00c860-a034-406c-b502-fd026772c38b admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:29.788124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00c860-a034-406c-b502-fd026772c38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:29.788413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00c860-a034-406c-b502-fd026772c38b admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:29.789110 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00c860-a034-406c-b502-fd026772c38b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:05:29.789411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00c860-a034-406c-b502-fd026772c38b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:29.791162 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00c860-a034-406c-b502-fd026772c38b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:29.805520 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1337/2668] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:05:29.809930 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcfebf-107e-49a3-ad10-255cd2e0613b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:29.820251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcfebf-107e-49a3-ad10-255cd2e0613b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:29.825135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f6976-e5ca-488b-a85c-073e0176287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:29.825756 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f6976-e5ca-488b-a85c-073e0176287a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:29.826039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f6976-e5ca-488b-a85c-073e0176287a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:29.826318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f6976-e5ca-488b-a85c-073e0176287a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:29.827040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f6976-e5ca-488b-a85c-073e0176287a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:29.829016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f6976-e5ca-488b-a85c-073e0176287a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:29.861674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1332/2669] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:05:29 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:05:29.866386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00c860-a034-406c-b502-fd026772c38b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:29.878847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00c860-a034-406c-b502-fd026772c38b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:29.939148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d717bf-784f-4cd6-bbf0-c2e72476c4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:29.940395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d717bf-784f-4cd6-bbf0-c2e72476c4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:29.940678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d717bf-784f-4cd6-bbf0-c2e72476c4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:29.940948 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d717bf-784f-4cd6-bbf0-c2e72476c4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:29.941618 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d717bf-784f-4cd6-bbf0-c2e72476c4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:29.943569 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d717bf-784f-4cd6-bbf0-c2e72476c4a9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:29.966148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1338/2670] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:05:29 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:29.971039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f6976-e5ca-488b-a85c-073e0176287a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:29.981519 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f6976-e5ca-488b-a85c-073e0176287a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:29.986994 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8f72a-0f92-4479-a237-6dcc921aadc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:29.987574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8f72a-0f92-4479-a237-6dcc921aadc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:29.987856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8f72a-0f92-4479-a237-6dcc921aadc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:29.988129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8f72a-0f92-4479-a237-6dcc921aadc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:29.988823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8f72a-0f92-4479-a237-6dcc921aadc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:29.990417 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8f72a-0f92-4479-a237-6dcc921aadc8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.000691 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1333/2671] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:30.004955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d717bf-784f-4cd6-bbf0-c2e72476c4a9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:30.015454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d717bf-784f-4cd6-bbf0-c2e72476c4a9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:30.020729 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7f799-b7b9-4b1e-84f9-4cfe6faa7ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:30.021349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7f799-b7b9-4b1e-84f9-4cfe6faa7ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.021633 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7f799-b7b9-4b1e-84f9-4cfe6faa7ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.021940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7f799-b7b9-4b1e-84f9-4cfe6faa7ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.022671 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7f799-b7b9-4b1e-84f9-4cfe6faa7ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:30.024336 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7f799-b7b9-4b1e-84f9-4cfe6faa7ffa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.036606 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1339/2672] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:30.040961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8f72a-0f92-4479-a237-6dcc921aadc8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:30.051520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8f72a-0f92-4479-a237-6dcc921aadc8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:30.058507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a8da9-cb5d-463d-b35d-458c30934423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:30.059245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a8da9-cb5d-463d-b35d-458c30934423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.059595 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a8da9-cb5d-463d-b35d-458c30934423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.059939 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a8da9-cb5d-463d-b35d-458c30934423 admin admin] PATH_INFO: `/v3/projects/84a7143cdd2a4577a21f419796ec4898/users/520fa4900ae040858dd3d1c3dda00302/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.072512 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:30.076786 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a8da9-cb5d-463d-b35d-458c30934423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a7143cdd2a4577a21f419796ec4898, user_id=520fa4900ae040858dd3d1c3dda00302, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:30.081380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a8da9-cb5d-463d-b35d-458c30934423 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.095359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1334/2673] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:30 2022] PUT /identity/v3/projects/84a7143cdd2a4577a21f419796ec4898/users/520fa4900ae040858dd3d1c3dda00302/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:30.105575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564b08f-f122-40d0-9c13-7f06484e05d6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.105941 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564b08f-f122-40d0-9c13-7f06484e05d6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.106290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564b08f-f122-40d0-9c13-7f06484e05d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.135012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f564b08f-f122-40d0-9c13-7f06484e05d6 None None] MFA Rules not processed for user `520fa4900ae040858dd3d1c3dda00302`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:30.136387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f564b08f-f122-40d0-9c13-7f06484e05d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:30.203811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1340/2674] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:30.208280 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a8da9-cb5d-463d-b35d-458c30934423 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:30.272549 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a8da9-cb5d-463d-b35d-458c30934423 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:30.277358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d6974-8346-4ae3-b70e-f265bdd2b61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:30.277984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d6974-8346-4ae3-b70e-f265bdd2b61b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.278268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d6974-8346-4ae3-b70e-f265bdd2b61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.278586 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d6974-8346-4ae3-b70e-f265bdd2b61b admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.279341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d6974-8346-4ae3-b70e-f265bdd2b61b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:30.281298 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d6974-8346-4ae3-b70e-f265bdd2b61b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.309555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1335/2675] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:05:30 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:05:30.313132 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564b08f-f122-40d0-9c13-7f06484e05d6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:30.325044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564b08f-f122-40d0-9c13-7f06484e05d6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:30.372530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:30.388741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c812b7a-622a-48d3-ab5c-8a82c324c62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:30.389371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c812b7a-622a-48d3-ab5c-8a82c324c62a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.389653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c812b7a-622a-48d3-ab5c-8a82c324c62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.390041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c812b7a-622a-48d3-ab5c-8a82c324c62a admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.390698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c812b7a-622a-48d3-ab5c-8a82c324c62a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:30.392687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c812b7a-622a-48d3-ab5c-8a82c324c62a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.417187 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1341/2676] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:05:30 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:30.418614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d6974-8346-4ae3-b70e-f265bdd2b61b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:30.432037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d6974-8346-4ae3-b70e-f265bdd2b61b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:30.437627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb04836-3c61-4700-b774-737246db25cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:30.438328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb04836-3c61-4700-b774-737246db25cf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.438734 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb04836-3c61-4700-b774-737246db25cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.439904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb04836-3c61-4700-b774-737246db25cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.440828 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb04836-3c61-4700-b774-737246db25cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:30.442933 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb04836-3c61-4700-b774-737246db25cf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.455454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1336/2677] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:30.459352 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c812b7a-622a-48d3-ab5c-8a82c324c62a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:30.472071 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c812b7a-622a-48d3-ab5c-8a82c324c62a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:30.476915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d1823-0cb4-4c46-ba4d-7b9f1575889a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:30.477609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d1823-0cb4-4c46-ba4d-7b9f1575889a admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.477894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d1823-0cb4-4c46-ba4d-7b9f1575889a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.478170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d1823-0cb4-4c46-ba4d-7b9f1575889a admin admin] PATH_INFO: `/v3/projects/35f84a50bc5b462e83ea43917b497dc5/users/6aad9cb5738044b797740881b5daa31a/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.497367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d1823-0cb4-4c46-ba4d-7b9f1575889a admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f84a50bc5b462e83ea43917b497dc5, user_id=6aad9cb5738044b797740881b5daa31a, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:30.500156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d1823-0cb4-4c46-ba4d-7b9f1575889a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.516207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1342/2678] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:30 2022] PUT /identity/v3/projects/35f84a50bc5b462e83ea43917b497dc5/users/6aad9cb5738044b797740881b5daa31a/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:30.520386 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb04836-3c61-4700-b774-737246db25cf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:30.530868 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb04836-3c61-4700-b774-737246db25cf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:30.611916 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e6eac-bced-48b2-aee2-2ee569e54559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:30.612960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e6eac-bced-48b2-aee2-2ee569e54559 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.613315 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e6eac-bced-48b2-aee2-2ee569e54559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.613667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e6eac-bced-48b2-aee2-2ee569e54559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.614501 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e6eac-bced-48b2-aee2-2ee569e54559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:30.616915 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e6eac-bced-48b2-aee2-2ee569e54559 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.634490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1337/2679] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:05:30.637688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d1823-0cb4-4c46-ba4d-7b9f1575889a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:30.650215 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d1823-0cb4-4c46-ba4d-7b9f1575889a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:30.655316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a77c2-f3e7-451a-880a-d7ca9b44baab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:30.656015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a77c2-f3e7-451a-880a-d7ca9b44baab admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.656296 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a77c2-f3e7-451a-880a-d7ca9b44baab admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.656575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a77c2-f3e7-451a-880a-d7ca9b44baab admin admin] PATH_INFO: `/v3/projects/35f84a50bc5b462e83ea43917b497dc5/users/6aad9cb5738044b797740881b5daa31a/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.660314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a77c2-f3e7-451a-880a-d7ca9b44baab admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f84a50bc5b462e83ea43917b497dc5, user_id=6aad9cb5738044b797740881b5daa31a, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:30.663013 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a77c2-f3e7-451a-880a-d7ca9b44baab admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.682129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1343/2680] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:30 2022] PUT /identity/v3/projects/35f84a50bc5b462e83ea43917b497dc5/users/6aad9cb5738044b797740881b5daa31a/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:30.694285 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f2fda-e3ae-4a74-8f0a-3d751b4dc137 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.696442 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f2fda-e3ae-4a74-8f0a-3d751b4dc137 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.696807 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f2fda-e3ae-4a74-8f0a-3d751b4dc137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.734069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d13f2fda-e3ae-4a74-8f0a-3d751b4dc137 None None] MFA Rules not processed for user `6aad9cb5738044b797740881b5daa31a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:30.735359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d13f2fda-e3ae-4a74-8f0a-3d751b4dc137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:30.796188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:30.820228 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1338/2681] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:30.831847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a77c2-f3e7-451a-880a-d7ca9b44baab admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:30.834290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3c6a77c2-f3e7-451a-880a-d7ca9b44baab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:30.902205 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a77c2-f3e7-451a-880a-d7ca9b44baab admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:30.905996 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522492f-e08c-4766-9b98-aa1a32b6dac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:30.906669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522492f-e08c-4766-9b98-aa1a32b6dac2 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:30.906950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522492f-e08c-4766-9b98-aa1a32b6dac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:30.907228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522492f-e08c-4766-9b98-aa1a32b6dac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:30.918534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522492f-e08c-4766-9b98-aa1a32b6dac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:30.920280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522492f-e08c-4766-9b98-aa1a32b6dac2 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:30.931646 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1344/2682] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:05:31.045070 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f2fda-e3ae-4a74-8f0a-3d751b4dc137 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:31.056110 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f2fda-e3ae-4a74-8f0a-3d751b4dc137 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:31.059847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8abcac-0431-4726-be4b-44ab1310249a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:31.060475 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8abcac-0431-4726-be4b-44ab1310249a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:31.060758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8abcac-0431-4726-be4b-44ab1310249a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:31.061032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8abcac-0431-4726-be4b-44ab1310249a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:31.123314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8abcac-0431-4726-be4b-44ab1310249a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:31.124688 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8abcac-0431-4726-be4b-44ab1310249a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:31.134640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1339/2683] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:32.843573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522492f-e08c-4766-9b98-aa1a32b6dac2 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:32.848005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c522492f-e08c-4766-9b98-aa1a32b6dac2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:32.928574 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522492f-e08c-4766-9b98-aa1a32b6dac2 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:32.933613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:32.934464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:32.934819 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:32.935091 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] PATH_INFO: `/v3/users/b1a98edf8e5a4a22b3ae7d041b582b2d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:32.936306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a98edf8e5a4a22b3ae7d041b582b2d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:32.938320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:32.994953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:32.998523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a98edf8e5a4a22b3ae7d041b582b2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.014861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a98edf8e5a4a22b3ae7d041b582b2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.015585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a98edf8e5a4a22b3ae7d041b582b2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.016179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a98edf8e5a4a22b3ae7d041b582b2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.030527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a98edf8e5a4a22b3ae7d041b582b2d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.046908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1345/2684] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:32 2022] DELETE /identity/v3/users/b1a98edf8e5a4a22b3ae7d041b582b2d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:33.224398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8abcac-0431-4726-be4b-44ab1310249a service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:33.239520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8abcac-0431-4726-be4b-44ab1310249a service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:33.299138 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:33.299713 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:33.299997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:33.300275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] PATH_INFO: `/v3/projects/76df23454a3c4a9bb4f61b044ed5a426` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:33.301432 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=76df23454a3c4a9bb4f61b044ed5a426)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:33.303431 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:33.349534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76df23454a3c4a9bb4f61b044ed5a426', 'request_id': 'req-70ac63da-10f4-4b63-9453-7eec9d4548ac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.357054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76df23454a3c4a9bb4f61b044ed5a426', 'request_id': 'req-70ac63da-10f4-4b63-9453-7eec9d4548ac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.362926 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76df23454a3c4a9bb4f61b044ed5a426', 'request_id': 'req-70ac63da-10f4-4b63-9453-7eec9d4548ac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.373760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] The token cache is being invalidate because project 76df23454a3c4a9bb4f61b044ed5a426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:33.374046 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.374802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.376892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1340/2685] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:33 2022] DELETE /identity/v3/projects/76df23454a3c4a9bb4f61b044ed5a426 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:33.381058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:33.382607 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:33.396467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a3039-266f-40a1-9510-09f151cef95a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:33.451279 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:33.451913 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:33.452188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:33.452455 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] PATH_INFO: `/v3/users/73a89bef4f814971aa72a1be13eedf63` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:33.453613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a89bef4f814971aa72a1be13eedf63)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:33.456347 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:33.502988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a89bef4f814971aa72a1be13eedf63'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.509938 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a89bef4f814971aa72a1be13eedf63'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.510684 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a89bef4f814971aa72a1be13eedf63'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.511292 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a89bef4f814971aa72a1be13eedf63'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.523325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a89bef4f814971aa72a1be13eedf63'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.536468 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1346/2686] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:33 2022] DELETE /identity/v3/users/73a89bef4f814971aa72a1be13eedf63 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:33.707051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:33.708518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:33.724587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac63da-10f4-4b63-9453-7eec9d4548ac admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:33.735726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:33.785063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:33.785642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:33.785902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:33.786171 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] PATH_INFO: `/v3/projects/f96b2dec79d34bc2be795eebfe1b9732` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:33.787338 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96b2dec79d34bc2be795eebfe1b9732)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:33.789229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:33.815674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:33.817758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:33.830910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eee76-3cf3-484c-8173-176e52c50d45 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:33.840474 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96b2dec79d34bc2be795eebfe1b9732', 'request_id': 'req-c58b4f78-ffc9-4169-bcba-0b2a115a284d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.849709 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96b2dec79d34bc2be795eebfe1b9732', 'request_id': 'req-c58b4f78-ffc9-4169-bcba-0b2a115a284d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.856645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96b2dec79d34bc2be795eebfe1b9732', 'request_id': 'req-c58b4f78-ffc9-4169-bcba-0b2a115a284d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.869613 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] The token cache is being invalidate because project f96b2dec79d34bc2be795eebfe1b9732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:33.869983 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.870730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c58b4f78-ffc9-4169-bcba-0b2a115a284d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.872746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1341/2687] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:33 2022] DELETE /identity/v3/projects/f96b2dec79d34bc2be795eebfe1b9732 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:33.873208 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:33.904318 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:33.904990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:33.905268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:33.905627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] PATH_INFO: `/v3/users/520fa4900ae040858dd3d1c3dda00302` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:33.906921 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] RBAC: Authorizing `identity:delete_user(user_id=520fa4900ae040858dd3d1c3dda00302)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:33.910128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:33.963862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '520fa4900ae040858dd3d1c3dda00302'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.972508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '520fa4900ae040858dd3d1c3dda00302'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.973251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '520fa4900ae040858dd3d1c3dda00302'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.973854 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '520fa4900ae040858dd3d1c3dda00302'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:33.987380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '520fa4900ae040858dd3d1c3dda00302'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.002334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1347/2688] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:33 2022] DELETE /identity/v3/users/520fa4900ae040858dd3d1c3dda00302 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:34.009730 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276944f2-57e7-4f92-8f29-4f95cee9b005 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:34.010012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276944f2-57e7-4f92-8f29-4f95cee9b005 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:34.010296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276944f2-57e7-4f92-8f29-4f95cee9b005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:34.030999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-276944f2-57e7-4f92-8f29-4f95cee9b005 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:34.032819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-276944f2-57e7-4f92-8f29-4f95cee9b005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:34.098008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1342/2689] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:34.108155 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:34.109602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:34.168254 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a1b8c-f0e1-4c6d-9def-f563d4397eff admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:34.172148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e618f-455e-4713-a2c1-20063e998881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:34.172756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e618f-455e-4713-a2c1-20063e998881 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:34.173097 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e618f-455e-4713-a2c1-20063e998881 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:34.173559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e618f-455e-4713-a2c1-20063e998881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:34.182316 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e618f-455e-4713-a2c1-20063e998881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:34.183763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e618f-455e-4713-a2c1-20063e998881 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:34.193847 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1348/2690] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:34.464303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276944f2-57e7-4f92-8f29-4f95cee9b005 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:34.465792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-276944f2-57e7-4f92-8f29-4f95cee9b005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:34.478782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276944f2-57e7-4f92-8f29-4f95cee9b005 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:34.483721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:34.484848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:34.485304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:34.486169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] PATH_INFO: `/v3/projects/84a7143cdd2a4577a21f419796ec4898` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:34.487398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a7143cdd2a4577a21f419796ec4898)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:34.489360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:34.533874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:34.542460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a7143cdd2a4577a21f419796ec4898', 'request_id': 'req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.551737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a7143cdd2a4577a21f419796ec4898', 'request_id': 'req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.557322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a7143cdd2a4577a21f419796ec4898', 'request_id': 'req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.570025 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] The token cache is being invalidate because project 84a7143cdd2a4577a21f419796ec4898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:34.570344 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.571023 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.572607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1343/2691] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:34 2022] DELETE /identity/v3/projects/84a7143cdd2a4577a21f419796ec4898 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:34.576610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e618f-455e-4713-a2c1-20063e998881 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:34.590005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e618f-455e-4713-a2c1-20063e998881 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:34.658828 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:34.659581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:34.659704 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:34.659963 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] PATH_INFO: `/v3/users/6aad9cb5738044b797740881b5daa31a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:34.661145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aad9cb5738044b797740881b5daa31a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:34.663088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:34.665458 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:34.714830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aad9cb5738044b797740881b5daa31a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.721954 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aad9cb5738044b797740881b5daa31a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.722705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aad9cb5738044b797740881b5daa31a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.723310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aad9cb5738044b797740881b5daa31a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.735317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c13875b3-fba4-4076-a199-0d4c9c7dbb6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aad9cb5738044b797740881b5daa31a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:34.751130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1349/2692] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:34 2022] DELETE /identity/v3/users/6aad9cb5738044b797740881b5daa31a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:35.059201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:35.061067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:35.077562 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0d3a0-8124-4022-9e97-9fc07ccab70e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:35.151980 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:35.152985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.153461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.153921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] PATH_INFO: `/v3/projects/35f84a50bc5b462e83ea43917b497dc5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.155889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f84a50bc5b462e83ea43917b497dc5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:35.158086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:35.208342 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f84a50bc5b462e83ea43917b497dc5', 'request_id': 'req-d2a88228-9254-4e8e-a8f0-69cebe82ae96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:35.216134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f84a50bc5b462e83ea43917b497dc5', 'request_id': 'req-d2a88228-9254-4e8e-a8f0-69cebe82ae96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:35.223035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f84a50bc5b462e83ea43917b497dc5', 'request_id': 'req-d2a88228-9254-4e8e-a8f0-69cebe82ae96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:35.236284 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] The token cache is being invalidate because project 35f84a50bc5b462e83ea43917b497dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:35.236815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:35.237510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2a88228-9254-4e8e-a8f0-69cebe82ae96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:35.239626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1344/2693] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:35 2022] DELETE /identity/v3/projects/35f84a50bc5b462e83ea43917b497dc5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:35.244887 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a64b8-fe31-448e-b9ad-2432a305d735 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.245360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a64b8-fe31-448e-b9ad-2432a305d735 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.245588 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a64b8-fe31-448e-b9ad-2432a305d735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.282584 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-809a64b8-fe31-448e-b9ad-2432a305d735 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:35.283810 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-809a64b8-fe31-448e-b9ad-2432a305d735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:35.344909 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1350/2694] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:35.352849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101187e-bb87-49b6-9a53-1dd45caa3303 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.353139 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101187e-bb87-49b6-9a53-1dd45caa3303 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.353416 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101187e-bb87-49b6-9a53-1dd45caa3303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.373704 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7101187e-bb87-49b6-9a53-1dd45caa3303 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:35.374936 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7101187e-bb87-49b6-9a53-1dd45caa3303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:35.383591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1345/2695] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:35.388164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a64b8-fe31-448e-b9ad-2432a305d735 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:35.398096 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a64b8-fe31-448e-b9ad-2432a305d735 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:35.402710 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9127c-48e4-4545-a6df-fb50a131b516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:35.403308 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9127c-48e4-4545-a6df-fb50a131b516 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.403582 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9127c-48e4-4545-a6df-fb50a131b516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.403850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9127c-48e4-4545-a6df-fb50a131b516 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.404538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9127c-48e4-4545-a6df-fb50a131b516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:05:35.404835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9127c-48e4-4545-a6df-fb50a131b516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:35.406528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9127c-48e4-4545-a6df-fb50a131b516 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:35.418180 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1351/2696] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:05:35.423494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101187e-bb87-49b6-9a53-1dd45caa3303 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:35.433453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101187e-bb87-49b6-9a53-1dd45caa3303 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:35.438207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ed181-4a04-4bf2-82c5-d5ffd32b8419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:35.439402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ed181-4a04-4bf2-82c5-d5ffd32b8419 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.439679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ed181-4a04-4bf2-82c5-d5ffd32b8419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.439967 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ed181-4a04-4bf2-82c5-d5ffd32b8419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.440615 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ed181-4a04-4bf2-82c5-d5ffd32b8419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:35.442548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ed181-4a04-4bf2-82c5-d5ffd32b8419 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:35.469799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1346/2697] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:05:35 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:05:35.474157 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9127c-48e4-4545-a6df-fb50a131b516 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:35.484464 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9127c-48e4-4545-a6df-fb50a131b516 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:35.541224 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74fed-f2fe-4910-888d-4afa809ef075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:35.541832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74fed-f2fe-4910-888d-4afa809ef075 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.542113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74fed-f2fe-4910-888d-4afa809ef075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.542374 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e74fed-f2fe-4910-888d-4afa809ef075 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.543060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e74fed-f2fe-4910-888d-4afa809ef075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:35.545288 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e74fed-f2fe-4910-888d-4afa809ef075 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:35.561568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:35.569878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1352/2698] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:05:35 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:05:35.573335 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ed181-4a04-4bf2-82c5-d5ffd32b8419 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:35.583836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ed181-4a04-4bf2-82c5-d5ffd32b8419 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:35.588489 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fc2c4-a15a-4344-bde7-5acf0a5a1109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:35.589093 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fc2c4-a15a-4344-bde7-5acf0a5a1109 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.589379 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fc2c4-a15a-4344-bde7-5acf0a5a1109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.589656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fc2c4-a15a-4344-bde7-5acf0a5a1109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.590351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fc2c4-a15a-4344-bde7-5acf0a5a1109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:35.591976 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fc2c4-a15a-4344-bde7-5acf0a5a1109 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:35.602459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1347/2699] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:35.606287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74fed-f2fe-4910-888d-4afa809ef075 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:35.617053 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e74fed-f2fe-4910-888d-4afa809ef075 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:35.621861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23dca3e-c018-4ced-b96a-85284f0b8de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:35.622600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23dca3e-c018-4ced-b96a-85284f0b8de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.622915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23dca3e-c018-4ced-b96a-85284f0b8de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.623298 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23dca3e-c018-4ced-b96a-85284f0b8de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.623973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23dca3e-c018-4ced-b96a-85284f0b8de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:35.625552 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23dca3e-c018-4ced-b96a-85284f0b8de3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:35.636860 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1353/2700] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:35.640887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fc2c4-a15a-4344-bde7-5acf0a5a1109 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:35.651032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fc2c4-a15a-4344-bde7-5acf0a5a1109 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:35.655729 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ee6b6-d2c8-48ef-bcdb-867ddafe0238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:35.656387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ee6b6-d2c8-48ef-bcdb-867ddafe0238 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.656679 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ee6b6-d2c8-48ef-bcdb-867ddafe0238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.656949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ee6b6-d2c8-48ef-bcdb-867ddafe0238 admin admin] PATH_INFO: `/v3/projects/5040831e390240669ac827b1e1c9220a/users/44b02661572040abb40f6b775e03e2e7/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.674180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ee6b6-d2c8-48ef-bcdb-867ddafe0238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5040831e390240669ac827b1e1c9220a, user_id=44b02661572040abb40f6b775e03e2e7, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:35.676888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ee6b6-d2c8-48ef-bcdb-867ddafe0238 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:35.679163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:35.691527 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1348/2701] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:35 2022] PUT /identity/v3/projects/5040831e390240669ac827b1e1c9220a/users/44b02661572040abb40f6b775e03e2e7/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:35.708979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8230e-23de-49f0-ae65-0e7dfa6e6caa None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.709263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8230e-23de-49f0-ae65-0e7dfa6e6caa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.709559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8230e-23de-49f0-ae65-0e7dfa6e6caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.739052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-61b8230e-23de-49f0-ae65-0e7dfa6e6caa None None] MFA Rules not processed for user `44b02661572040abb40f6b775e03e2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:35.740241 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-61b8230e-23de-49f0-ae65-0e7dfa6e6caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:35.804156 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1354/2702] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:35.808761 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ee6b6-d2c8-48ef-bcdb-867ddafe0238 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:35.873430 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ee6b6-d2c8-48ef-bcdb-867ddafe0238 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:35.878140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377d520-8e02-4c2c-b00d-fc507b41c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:35.878787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377d520-8e02-4c2c-b00d-fc507b41c9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.879075 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377d520-8e02-4c2c-b00d-fc507b41c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.879351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377d520-8e02-4c2c-b00d-fc507b41c9a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.880005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377d520-8e02-4c2c-b00d-fc507b41c9a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:35.881931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377d520-8e02-4c2c-b00d-fc507b41c9a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:35.908941 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1349/2703] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:05:35 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:35.912991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8230e-23de-49f0-ae65-0e7dfa6e6caa None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:35.924562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8230e-23de-49f0-ae65-0e7dfa6e6caa None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:35.980192 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936ac26-6d32-4923-a1e4-d79a4810f0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:35.980807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936ac26-6d32-4923-a1e4-d79a4810f0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:35.981084 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936ac26-6d32-4923-a1e4-d79a4810f0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:35.981356 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936ac26-6d32-4923-a1e4-d79a4810f0f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:35.982010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936ac26-6d32-4923-a1e4-d79a4810f0f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:35.983976 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936ac26-6d32-4923-a1e4-d79a4810f0f1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:36.005422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1355/2704] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:05:35 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:36.009296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377d520-8e02-4c2c-b00d-fc507b41c9a3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:36.020727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377d520-8e02-4c2c-b00d-fc507b41c9a3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:36.025822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acc859-6ff5-40b9-be27-e61353d187c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:36.026469 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acc859-6ff5-40b9-be27-e61353d187c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:36.026752 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acc859-6ff5-40b9-be27-e61353d187c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:36.027033 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acc859-6ff5-40b9-be27-e61353d187c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:36.027732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acc859-6ff5-40b9-be27-e61353d187c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:36.029340 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acc859-6ff5-40b9-be27-e61353d187c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:36.040802 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1350/2705] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:36.045244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936ac26-6d32-4923-a1e4-d79a4810f0f1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:36.055377 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936ac26-6d32-4923-a1e4-d79a4810f0f1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:36.060026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ed32c-8067-4f95-97a7-3216b3fb9a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:36.060688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ed32c-8067-4f95-97a7-3216b3fb9a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:36.060968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ed32c-8067-4f95-97a7-3216b3fb9a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:36.061243 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ed32c-8067-4f95-97a7-3216b3fb9a09 admin admin] PATH_INFO: `/v3/projects/166db4c77dde4e7384032e0ed2017ef0/users/10316533693543aca31ea53076fc6840/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:36.077075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ed32c-8067-4f95-97a7-3216b3fb9a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=166db4c77dde4e7384032e0ed2017ef0, user_id=10316533693543aca31ea53076fc6840, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:36.080578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ed32c-8067-4f95-97a7-3216b3fb9a09 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:36.093131 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1356/2706] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:36 2022] PUT /identity/v3/projects/166db4c77dde4e7384032e0ed2017ef0/users/10316533693543aca31ea53076fc6840/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:36.097702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acc859-6ff5-40b9-be27-e61353d187c3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:36.107797 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acc859-6ff5-40b9-be27-e61353d187c3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:36.164437 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d45e3-b67d-4402-ae70-accb6368b3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:36.165049 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d45e3-b67d-4402-ae70-accb6368b3ce admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:36.165334 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d45e3-b67d-4402-ae70-accb6368b3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:36.165609 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d45e3-b67d-4402-ae70-accb6368b3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:36.166295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d45e3-b67d-4402-ae70-accb6368b3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:36.168065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d45e3-b67d-4402-ae70-accb6368b3ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:36.178077 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1351/2707] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:36.182202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ed32c-8067-4f95-97a7-3216b3fb9a09 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:36.192496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ed32c-8067-4f95-97a7-3216b3fb9a09 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:36.197159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716c857-7fd8-4863-84dd-c37f9b2d3e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:36.197807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716c857-7fd8-4863-84dd-c37f9b2d3e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:36.198085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716c857-7fd8-4863-84dd-c37f9b2d3e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:36.198357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716c857-7fd8-4863-84dd-c37f9b2d3e9f admin admin] PATH_INFO: `/v3/projects/166db4c77dde4e7384032e0ed2017ef0/users/10316533693543aca31ea53076fc6840/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:36.200349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716c857-7fd8-4863-84dd-c37f9b2d3e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=166db4c77dde4e7384032e0ed2017ef0, user_id=10316533693543aca31ea53076fc6840, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:36.202995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716c857-7fd8-4863-84dd-c37f9b2d3e9f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:36.213525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1357/2708] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:36 2022] PUT /identity/v3/projects/166db4c77dde4e7384032e0ed2017ef0/users/10316533693543aca31ea53076fc6840/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:36.225426 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2fc01-f5b2-439f-bd65-2ce919c4138e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:36.225714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2fc01-f5b2-439f-bd65-2ce919c4138e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:36.225995 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2fc01-f5b2-439f-bd65-2ce919c4138e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:36.253636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-60b2fc01-f5b2-439f-bd65-2ce919c4138e None None] MFA Rules not processed for user `10316533693543aca31ea53076fc6840`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:36.254882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60b2fc01-f5b2-439f-bd65-2ce919c4138e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:36.308580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:36.327141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1352/2709] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:36.339447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716c857-7fd8-4863-84dd-c37f9b2d3e9f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:36.341333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f716c857-7fd8-4863-84dd-c37f9b2d3e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:36.374229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:36.408333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716c857-7fd8-4863-84dd-c37f9b2d3e9f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:36.411985 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43ac12-f49a-4173-aefd-bce3768913f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:36.412596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43ac12-f49a-4173-aefd-bce3768913f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:36.412872 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43ac12-f49a-4173-aefd-bce3768913f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:36.413139 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43ac12-f49a-4173-aefd-bce3768913f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:36.423088 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43ac12-f49a-4173-aefd-bce3768913f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:36.425010 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43ac12-f49a-4173-aefd-bce3768913f7 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:36.434870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1358/2710] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:36.522257 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2fc01-f5b2-439f-bd65-2ce919c4138e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:36.532065 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2fc01-f5b2-439f-bd65-2ce919c4138e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:36.537111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786a278-efc8-4d12-9646-19fc4912f174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:36.537732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786a278-efc8-4d12-9646-19fc4912f174 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:36.538017 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786a278-efc8-4d12-9646-19fc4912f174 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:36.538327 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786a278-efc8-4d12-9646-19fc4912f174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:36.600991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786a278-efc8-4d12-9646-19fc4912f174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:36.602453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786a278-efc8-4d12-9646-19fc4912f174 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:36.612209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1353/2711] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:41.402768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43ac12-f49a-4173-aefd-bce3768913f7 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:41.464793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43ac12-f49a-4173-aefd-bce3768913f7 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:41.469375 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:41.470007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:41.470286 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:41.470602 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] PATH_INFO: `/v3/users/44b02661572040abb40f6b775e03e2e7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:41.471870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b02661572040abb40f6b775e03e2e7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:41.473767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:41.519104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b02661572040abb40f6b775e03e2e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:41.525901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b02661572040abb40f6b775e03e2e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:41.526572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b02661572040abb40f6b775e03e2e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:41.527173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b02661572040abb40f6b775e03e2e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:41.538906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b02661572040abb40f6b775e03e2e7'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:41.552580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1359/2712] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:41 2022] DELETE /identity/v3/users/44b02661572040abb40f6b775e03e2e7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:41.557081 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf397d-2df3-44ea-a551-04a838cd65d8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:41.557503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf397d-2df3-44ea-a551-04a838cd65d8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:41.557614 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf397d-2df3-44ea-a551-04a838cd65d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:41.577195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fddf397d-2df3-44ea-a551-04a838cd65d8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:41.578410 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fddf397d-2df3-44ea-a551-04a838cd65d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:41.635343 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1354/2713] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:41.645235 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:41.646676 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:41.703115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18757b-251d-438b-8c76-54061626f726 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:41.706971 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a7f0c-75e2-40ee-b1f2-ccee7b3c14e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:41.707578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a7f0c-75e2-40ee-b1f2-ccee7b3c14e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:41.707850 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a7f0c-75e2-40ee-b1f2-ccee7b3c14e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:41.708117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a7f0c-75e2-40ee-b1f2-ccee7b3c14e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:41.713919 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:41.718825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a7f0c-75e2-40ee-b1f2-ccee7b3c14e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:41.720583 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a7f0c-75e2-40ee-b1f2-ccee7b3c14e9 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:41.730928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1360/2714] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:41.998573 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf397d-2df3-44ea-a551-04a838cd65d8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:42.000002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fddf397d-2df3-44ea-a551-04a838cd65d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:42.014195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf397d-2df3-44ea-a551-04a838cd65d8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:42.025502 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:42.026163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:42.026464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:42.026749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] PATH_INFO: `/v3/projects/5040831e390240669ac827b1e1c9220a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:42.028006 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5040831e390240669ac827b1e1c9220a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:42.029927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:42.078892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5040831e390240669ac827b1e1c9220a', 'request_id': 'req-55c8be30-30ab-411e-baae-db57a599cd9b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.086164 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5040831e390240669ac827b1e1c9220a', 'request_id': 'req-55c8be30-30ab-411e-baae-db57a599cd9b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.091751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5040831e390240669ac827b1e1c9220a', 'request_id': 'req-55c8be30-30ab-411e-baae-db57a599cd9b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.103223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] The token cache is being invalidate because project 5040831e390240669ac827b1e1c9220a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:42.103510 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.104126 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.105605 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1355/2715] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:41 2022] DELETE /identity/v3/projects/5040831e390240669ac827b1e1c9220a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:42.109581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a7f0c-75e2-40ee-b1f2-ccee7b3c14e9 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:42.119926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a7f0c-75e2-40ee-b1f2-ccee7b3c14e9 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:42.177260 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:42.177926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:42.178200 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:42.178503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] PATH_INFO: `/v3/users/10316533693543aca31ea53076fc6840` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:42.182290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=10316533693543aca31ea53076fc6840)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:42.184268 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:42.241325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10316533693543aca31ea53076fc6840'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.252443 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10316533693543aca31ea53076fc6840'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.253312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10316533693543aca31ea53076fc6840'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.254009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10316533693543aca31ea53076fc6840'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.266108 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-da6d365f-8092-4b31-899d-820a25cac07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10316533693543aca31ea53076fc6840'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.286683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1361/2716] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:42 2022] DELETE /identity/v3/users/10316533693543aca31ea53076fc6840 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:42.567407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:42.568999 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:42.585196 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8be30-30ab-411e-baae-db57a599cd9b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:42.595362 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:42.648817 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:42.649486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:42.649763 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:42.650032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] PATH_INFO: `/v3/projects/166db4c77dde4e7384032e0ed2017ef0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:42.651204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=166db4c77dde4e7384032e0ed2017ef0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:42.653140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:42.699384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '166db4c77dde4e7384032e0ed2017ef0', 'request_id': 'req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.707292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '166db4c77dde4e7384032e0ed2017ef0', 'request_id': 'req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.712852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '166db4c77dde4e7384032e0ed2017ef0', 'request_id': 'req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.723767 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] The token cache is being invalidate because project 166db4c77dde4e7384032e0ed2017ef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:42.724059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.724690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc1ae7d5-0de4-47ea-a3a7-f7ba21ab3db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:42.726214 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1356/2717] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:42 2022] DELETE /identity/v3/projects/166db4c77dde4e7384032e0ed2017ef0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:42.731290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e26338-613c-421e-a97c-8ab32fc8e241 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:42.731929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e26338-613c-421e-a97c-8ab32fc8e241 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:42.731929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e26338-613c-421e-a97c-8ab32fc8e241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:42.751277 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f9e26338-613c-421e-a97c-8ab32fc8e241 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:42.753228 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f9e26338-613c-421e-a97c-8ab32fc8e241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:42.810740 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1362/2718] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:42.819152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f91d3-e225-44b9-a93f-48c400405304 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:42.819444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f91d3-e225-44b9-a93f-48c400405304 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:42.819725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f91d3-e225-44b9-a93f-48c400405304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:42.841218 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-557f91d3-e225-44b9-a93f-48c400405304 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:42.842370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-557f91d3-e225-44b9-a93f-48c400405304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:42.851213 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1357/2719] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:42.855915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e26338-613c-421e-a97c-8ab32fc8e241 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:42.866679 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e26338-613c-421e-a97c-8ab32fc8e241 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:42.871312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ca0b2-e744-4271-b21c-eb8f4bf2eb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:42.871926 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ca0b2-e744-4271-b21c-eb8f4bf2eb03 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:42.872207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ca0b2-e744-4271-b21c-eb8f4bf2eb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:42.872483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ca0b2-e744-4271-b21c-eb8f4bf2eb03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:42.873186 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ca0b2-e744-4271-b21c-eb8f4bf2eb03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:05:42.873492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ca0b2-e744-4271-b21c-eb8f4bf2eb03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:42.875128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ca0b2-e744-4271-b21c-eb8f4bf2eb03 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:42.883997 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:42.887545 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1363/2720] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:05:42.891727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f91d3-e225-44b9-a93f-48c400405304 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:42.901132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f91d3-e225-44b9-a93f-48c400405304 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:42.906037 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961099e-8833-47cc-8031-8a85a054b7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:42.906683 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961099e-8833-47cc-8031-8a85a054b7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:42.906964 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961099e-8833-47cc-8031-8a85a054b7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:42.907234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961099e-8833-47cc-8031-8a85a054b7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:42.907892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961099e-8833-47cc-8031-8a85a054b7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:42.910307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961099e-8833-47cc-8031-8a85a054b7b6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:42.938085 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1358/2721] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:05:42 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:42.942608 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ca0b2-e744-4271-b21c-eb8f4bf2eb03 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:42.952670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ca0b2-e744-4271-b21c-eb8f4bf2eb03 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:43.010688 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e48c1-d1c7-4fd1-865b-140e868e4a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:43.011310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e48c1-d1c7-4fd1-865b-140e868e4a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:43.011589 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e48c1-d1c7-4fd1-865b-140e868e4a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:43.011861 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e48c1-d1c7-4fd1-865b-140e868e4a49 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:43.012538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e48c1-d1c7-4fd1-865b-140e868e4a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:43.014533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e48c1-d1c7-4fd1-865b-140e868e4a49 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:43.037739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1364/2722] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:05:42 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:05:43.041938 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961099e-8833-47cc-8031-8a85a054b7b6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:43.052589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961099e-8833-47cc-8031-8a85a054b7b6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:43.057303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def67df-423a-4d84-9afd-9401a18a8150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:43.057907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def67df-423a-4d84-9afd-9401a18a8150 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:43.058188 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def67df-423a-4d84-9afd-9401a18a8150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:43.058484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def67df-423a-4d84-9afd-9401a18a8150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:43.059185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def67df-423a-4d84-9afd-9401a18a8150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:43.060779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def67df-423a-4d84-9afd-9401a18a8150 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:43.071223 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1359/2723] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:43.075058 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e48c1-d1c7-4fd1-865b-140e868e4a49 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:43.085653 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e48c1-d1c7-4fd1-865b-140e868e4a49 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:43.090305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3bcec2-46cc-4f86-ba8b-d083afa9fb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:43.090943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bcec2-46cc-4f86-ba8b-d083afa9fb84 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:43.091274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bcec2-46cc-4f86-ba8b-d083afa9fb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:43.091493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3bcec2-46cc-4f86-ba8b-d083afa9fb84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:43.092181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bcec2-46cc-4f86-ba8b-d083afa9fb84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:43.093760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3bcec2-46cc-4f86-ba8b-d083afa9fb84 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:43.104951 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1365/2724] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:43.109035 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def67df-423a-4d84-9afd-9401a18a8150 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:43.118960 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def67df-423a-4d84-9afd-9401a18a8150 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:43.123706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96ca5b-8198-4267-aee1-679e5cb79803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:43.124356 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96ca5b-8198-4267-aee1-679e5cb79803 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:43.124639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96ca5b-8198-4267-aee1-679e5cb79803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:43.124914 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96ca5b-8198-4267-aee1-679e5cb79803 admin admin] PATH_INFO: `/v3/projects/9e93d7ae01c94657ab3a2ad37a8ceb7b/users/719d32b1c49c4db49fb44c29567805a5/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:43.139472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96ca5b-8198-4267-aee1-679e5cb79803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e93d7ae01c94657ab3a2ad37a8ceb7b, user_id=719d32b1c49c4db49fb44c29567805a5, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:43.142149 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96ca5b-8198-4267-aee1-679e5cb79803 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:43.155322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1360/2725] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:43 2022] PUT /identity/v3/projects/9e93d7ae01c94657ab3a2ad37a8ceb7b/users/719d32b1c49c4db49fb44c29567805a5/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:43.222880 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302c07b-5963-4136-8e4b-d66c12be967a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:43.223160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302c07b-5963-4136-8e4b-d66c12be967a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:43.223432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302c07b-5963-4136-8e4b-d66c12be967a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:43.253320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b302c07b-5963-4136-8e4b-d66c12be967a None None] MFA Rules not processed for user `719d32b1c49c4db49fb44c29567805a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:43.254579 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b302c07b-5963-4136-8e4b-d66c12be967a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:43.453427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1366/2726] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:43.467644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96ca5b-8198-4267-aee1-679e5cb79803 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:43.470030 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3d96ca5b-8198-4267-aee1-679e5cb79803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:43.477353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302c07b-5963-4136-8e4b-d66c12be967a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:43.479039 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b302c07b-5963-4136-8e4b-d66c12be967a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:43.512574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:43.529742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96ca5b-8198-4267-aee1-679e5cb79803 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:43.534123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5de3fb-409e-46a5-8d49-4233e2b1bfa2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '548acf58bf9140ec9f1ba126e10deb45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:43.534794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5de3fb-409e-46a5-8d49-4233e2b1bfa2 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:43.535074 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5de3fb-409e-46a5-8d49-4233e2b1bfa2 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:43.535354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5de3fb-409e-46a5-8d49-4233e2b1bfa2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:43.536255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302c07b-5963-4136-8e4b-d66c12be967a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:43.539798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098274a-4954-46b1-a26d-215a9ea24ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:43.540404 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098274a-4954-46b1-a26d-215a9ea24ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:43.540766 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098274a-4954-46b1-a26d-215a9ea24ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:43.540939 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098274a-4954-46b1-a26d-215a9ea24ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:43.542720 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8098274a-4954-46b1-a26d-215a9ea24ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:43.544114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5de3fb-409e-46a5-8d49-4233e2b1bfa2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:43.545840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5de3fb-409e-46a5-8d49-4233e2b1bfa2 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:43.555587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1361/2727] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:05:43.566459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:43.616845 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098274a-4954-46b1-a26d-215a9ea24ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:43.618388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098274a-4954-46b1-a26d-215a9ea24ba2 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:43.630259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1367/2728] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:05:47.271400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5de3fb-409e-46a5-8d49-4233e2b1bfa2 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:47.272861 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc5de3fb-409e-46a5-8d49-4233e2b1bfa2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:47.288364 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5de3fb-409e-46a5-8d49-4233e2b1bfa2 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:47.292874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:47.293509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:47.293787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:47.294755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] PATH_INFO: `/v3/users/d3a56c6d44b74429a6616cdfc2869b71` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:47.295579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a56c6d44b74429a6616cdfc2869b71)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:47.297503 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:47.355622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a56c6d44b74429a6616cdfc2869b71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.363969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a56c6d44b74429a6616cdfc2869b71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.364660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a56c6d44b74429a6616cdfc2869b71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.365449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a56c6d44b74429a6616cdfc2869b71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.377608 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-79344b57-0c77-4308-beb5-7dcec2c7cf3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a56c6d44b74429a6616cdfc2869b71'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.390727 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1362/2729] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:05:47 2022] DELETE /identity/v3/users/d3a56c6d44b74429a6616cdfc2869b71 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:47.597830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098274a-4954-46b1-a26d-215a9ea24ba2 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:47.620341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098274a-4954-46b1-a26d-215a9ea24ba2 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:47.697281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:47.698325 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:47.698796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:47.699257 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] PATH_INFO: `/v3/projects/bc6f8afc1abf427f92c44fdc55994b18` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:47.700592 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc6f8afc1abf427f92c44fdc55994b18)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:47.702868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:47.781153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc6f8afc1abf427f92c44fdc55994b18', 'request_id': 'req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.792366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc6f8afc1abf427f92c44fdc55994b18', 'request_id': 'req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.800663 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc6f8afc1abf427f92c44fdc55994b18', 'request_id': 'req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.813965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] The token cache is being invalidate because project bc6f8afc1abf427f92c44fdc55994b18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:05:47.813965 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.814901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-baac3be0-c4fd-44ab-bac1-ad6adc6bc7b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:05:47.821117 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1368/2730] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:05:47 2022] DELETE /identity/v3/projects/bc6f8afc1abf427f92c44fdc55994b18 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:47.823499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac92a14-c73c-4167-92d0-88af32fa905b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:47.823855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac92a14-c73c-4167-92d0-88af32fa905b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:47.824219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac92a14-c73c-4167-92d0-88af32fa905b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:47.845059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eac92a14-c73c-4167-92d0-88af32fa905b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:47.846887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eac92a14-c73c-4167-92d0-88af32fa905b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:47.922540 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1363/2731] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:05:47.929814 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9c431-f1a4-425f-9c93-6394d2a38236 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:47.930101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9c431-f1a4-425f-9c93-6394d2a38236 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:47.930378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9c431-f1a4-425f-9c93-6394d2a38236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:47.951726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3bf9c431-f1a4-425f-9c93-6394d2a38236 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:47.952929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3bf9c431-f1a4-425f-9c93-6394d2a38236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:47.961459 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1369/2732] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:47.965796 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac92a14-c73c-4167-92d0-88af32fa905b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:47.975640 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac92a14-c73c-4167-92d0-88af32fa905b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:47.980304 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382bcfb-494d-4584-8608-52fd5194ee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:47.980918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382bcfb-494d-4584-8608-52fd5194ee78 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:47.981195 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382bcfb-494d-4584-8608-52fd5194ee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:47.981463 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382bcfb-494d-4584-8608-52fd5194ee78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:47.982157 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382bcfb-494d-4584-8608-52fd5194ee78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:05:47.982478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382bcfb-494d-4584-8608-52fd5194ee78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:47.984080 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382bcfb-494d-4584-8608-52fd5194ee78 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:47.996317 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1364/2733] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:05:48.000728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9c431-f1a4-425f-9c93-6394d2a38236 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:48.011094 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9c431-f1a4-425f-9c93-6394d2a38236 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:48.016420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ce72e-eeae-4718-affd-efe740fd564e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:48.017045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ce72e-eeae-4718-affd-efe740fd564e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:48.018383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ce72e-eeae-4718-affd-efe740fd564e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:48.018848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ce72e-eeae-4718-affd-efe740fd564e admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:48.020198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ce72e-eeae-4718-affd-efe740fd564e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:48.023111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ce72e-eeae-4718-affd-efe740fd564e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:48.057610 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1370/2734] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:05:47 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:05:48.058886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382bcfb-494d-4584-8608-52fd5194ee78 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:48.070121 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382bcfb-494d-4584-8608-52fd5194ee78 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:48.120088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:48.145402 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84619228-2ce9-4858-bb82-0c144a4f8386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:48.146133 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84619228-2ce9-4858-bb82-0c144a4f8386 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:48.146887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84619228-2ce9-4858-bb82-0c144a4f8386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:48.146887 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84619228-2ce9-4858-bb82-0c144a4f8386 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:48.147597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84619228-2ce9-4858-bb82-0c144a4f8386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:48.149617 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84619228-2ce9-4858-bb82-0c144a4f8386 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:48.192766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1365/2735] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:05:48 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:05:48.196756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ce72e-eeae-4718-affd-efe740fd564e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:48.208824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ce72e-eeae-4718-affd-efe740fd564e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:48.214460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ff9b7-3dd5-493d-a3d8-ffc34909e982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:48.215185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ff9b7-3dd5-493d-a3d8-ffc34909e982 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:48.215530 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ff9b7-3dd5-493d-a3d8-ffc34909e982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:48.215875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ff9b7-3dd5-493d-a3d8-ffc34909e982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:48.216648 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ff9b7-3dd5-493d-a3d8-ffc34909e982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:48.218334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ff9b7-3dd5-493d-a3d8-ffc34909e982 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:48.234255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1371/2736] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:48.235784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84619228-2ce9-4858-bb82-0c144a4f8386 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:48.252526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84619228-2ce9-4858-bb82-0c144a4f8386 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:48.259886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e2f9c-9a84-4198-94bd-3858379e8991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:48.260380 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e2f9c-9a84-4198-94bd-3858379e8991 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:48.260667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e2f9c-9a84-4198-94bd-3858379e8991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:48.261358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e2f9c-9a84-4198-94bd-3858379e8991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:48.261758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e2f9c-9a84-4198-94bd-3858379e8991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:48.263478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e2f9c-9a84-4198-94bd-3858379e8991 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:48.289824 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1366/2737] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:05:48.293917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ff9b7-3dd5-493d-a3d8-ffc34909e982 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:48.306877 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ff9b7-3dd5-493d-a3d8-ffc34909e982 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:48.318673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddc4c0-8e35-430c-8ce9-b1f609765cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:48.318673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc4c0-8e35-430c-8ce9-b1f609765cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:48.318673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc4c0-8e35-430c-8ce9-b1f609765cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:48.318673 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc4c0-8e35-430c-8ce9-b1f609765cd6 admin admin] PATH_INFO: `/v3/projects/d9f8c70dc5de488698a8ab775c618088/users/3a805f5dec66484c8d950b6bc52e3e03/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:48.319064 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:48.332669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddc4c0-8e35-430c-8ce9-b1f609765cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f8c70dc5de488698a8ab775c618088, user_id=3a805f5dec66484c8d950b6bc52e3e03, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:48.335568 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddc4c0-8e35-430c-8ce9-b1f609765cd6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:48.351143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1372/2738] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:05:48 2022] PUT /identity/v3/projects/d9f8c70dc5de488698a8ab775c618088/users/3a805f5dec66484c8d950b6bc52e3e03/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:05:48.356635 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374448e-a58c-4799-98f4-3011c8b05b53 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:48.356942 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374448e-a58c-4799-98f4-3011c8b05b53 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:48.357219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374448e-a58c-4799-98f4-3011c8b05b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:48.383647 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d374448e-a58c-4799-98f4-3011c8b05b53 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:48.384845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d374448e-a58c-4799-98f4-3011c8b05b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:48.469969 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1367/2739] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 30 05:05:48.480188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddc4c0-8e35-430c-8ce9-b1f609765cd6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:48.481962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ceddc4c0-8e35-430c-8ce9-b1f609765cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:48.566614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddc4c0-8e35-430c-8ce9-b1f609765cd6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:48.570471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f30d7-6ab6-4109-8688-51464a91869e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:48.571217 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f30d7-6ab6-4109-8688-51464a91869e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:48.571491 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f30d7-6ab6-4109-8688-51464a91869e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:48.571758 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f30d7-6ab6-4109-8688-51464a91869e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:48.581718 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f30d7-6ab6-4109-8688-51464a91869e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:48.583481 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f30d7-6ab6-4109-8688-51464a91869e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:48.599383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1373/2740] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:05:53.415785 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc225d-e1f6-447c-83ec-640665420f9a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:53.416445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc225d-e1f6-447c-83ec-640665420f9a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:53.416445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc225d-e1f6-447c-83ec-640665420f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:53.456706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ebcc225d-e1f6-447c-83ec-640665420f9a None None] MFA Rules not processed for user `3a805f5dec66484c8d950b6bc52e3e03`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:53.458268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ebcc225d-e1f6-447c-83ec-640665420f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:53.541155 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1368/2741] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:53.557839 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f30d7-6ab6-4109-8688-51464a91869e service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:53.560060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-488f30d7-6ab6-4109-8688-51464a91869e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:53.642806 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f30d7-6ab6-4109-8688-51464a91869e service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:53.651196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779ef97-ca1e-49bf-b106-9122d76d01cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:53.651846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779ef97-ca1e-49bf-b106-9122d76d01cb service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:53.652124 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779ef97-ca1e-49bf-b106-9122d76d01cb service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:53.652399 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779ef97-ca1e-49bf-b106-9122d76d01cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:53.663251 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779ef97-ca1e-49bf-b106-9122d76d01cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:53.664950 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779ef97-ca1e-49bf-b106-9122d76d01cb service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:53.676119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1374/2742] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:05:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:05:54.093173 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60793b-0817-42da-b8fa-07f006683715 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:54.093804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60793b-0817-42da-b8fa-07f006683715 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:54.094111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60793b-0817-42da-b8fa-07f006683715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:54.115468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6b60793b-0817-42da-b8fa-07f006683715 None None] MFA Rules not processed for user `e0f2d5364cb84337b77956dc5da78b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:54.116843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6b60793b-0817-42da-b8fa-07f006683715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:54.168961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:05:54.184363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1369/2743] 10.222.0.20 () {58 vars in 967 bytes} [Sat Jul 30 05:05:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:05:54.209784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779ef97-ca1e-49bf-b106-9122d76d01cb service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:05:54.211317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4779ef97-ca1e-49bf-b106-9122d76d01cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:54.284360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779ef97-ca1e-49bf-b106-9122d76d01cb service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:05:54.289778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607c8e6-b73f-4ef6-969f-e1505aa815a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bda2af62ee4ee39a9a350b7ce6430e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:05:54.290409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607c8e6-b73f-4ef6-969f-e1505aa815a2 service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:54.290716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607c8e6-b73f-4ef6-969f-e1505aa815a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:54.290991 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607c8e6-b73f-4ef6-969f-e1505aa815a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:54.303878 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607c8e6-b73f-4ef6-969f-e1505aa815a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:05:54.306717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607c8e6-b73f-4ef6-969f-e1505aa815a2 service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:05:54.324448 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1375/2744] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Jul 30 05:05:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 30 05:05:55.129840 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae044df0-3c0f-47e1-a170-2964550b7af4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:05:55.130798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae044df0-3c0f-47e1-a170-2964550b7af4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:05:55.131746 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae044df0-3c0f-47e1-a170-2964550b7af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:05:55.152593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ae044df0-3c0f-47e1-a170-2964550b7af4 None None] MFA Rules not processed for user `3a805f5dec66484c8d950b6bc52e3e03`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:05:55.153844 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae044df0-3c0f-47e1-a170-2964550b7af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:05:55.161880 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1370/2745] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:05:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:06:00.648295 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607c8e6-b73f-4ef6-969f-e1505aa815a2 service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:00.649853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c607c8e6-b73f-4ef6-969f-e1505aa815a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:00.661577 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:00.667581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607c8e6-b73f-4ef6-969f-e1505aa815a2 service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:00.673577 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:00.674219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:00.674527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:00.674798 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] PATH_INFO: `/v3/users/719d32b1c49c4db49fb44c29567805a5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:00.675990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=719d32b1c49c4db49fb44c29567805a5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:00.678223 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:00.729656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '719d32b1c49c4db49fb44c29567805a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:00.736809 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '719d32b1c49c4db49fb44c29567805a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:00.737630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '719d32b1c49c4db49fb44c29567805a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:00.738243 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '719d32b1c49c4db49fb44c29567805a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:00.753529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '719d32b1c49c4db49fb44c29567805a5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:00.768258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1376/2746] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:06:00 2022] DELETE /identity/v3/users/719d32b1c49c4db49fb44c29567805a5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:06:00.773087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1747e07-f382-4d1a-860d-4c713dfa12c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:00.773777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1747e07-f382-4d1a-860d-4c713dfa12c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:00.773929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1747e07-f382-4d1a-860d-4c713dfa12c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:00.793080 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1747e07-f382-4d1a-860d-4c713dfa12c8 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:06:00.794632 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1747e07-f382-4d1a-860d-4c713dfa12c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:00.861695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1371/2747] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:06:00.868487 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:00.869911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:00.928853 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1fd05-ebcd-4dc2-9703-c122c20522a3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:00.932501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf36890-3fb9-4e4d-b34c-20b3fd95123e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:00.933118 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36890-3fb9-4e4d-b34c-20b3fd95123e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:00.933394 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36890-3fb9-4e4d-b34c-20b3fd95123e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:00.933659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36890-3fb9-4e4d-b34c-20b3fd95123e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:00.943314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf36890-3fb9-4e4d-b34c-20b3fd95123e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:00.944749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf36890-3fb9-4e4d-b34c-20b3fd95123e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:00.954160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1377/2748] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:06:01.252865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1747e07-f382-4d1a-860d-4c713dfa12c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:01.254773 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1747e07-f382-4d1a-860d-4c713dfa12c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:01.274495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1747e07-f382-4d1a-860d-4c713dfa12c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:01.279865 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:01.280523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.280825 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.281089 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] PATH_INFO: `/v3/projects/9e93d7ae01c94657ab3a2ad37a8ceb7b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.282874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e93d7ae01c94657ab3a2ad37a8ceb7b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:01.285016 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:01.337041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e93d7ae01c94657ab3a2ad37a8ceb7b', 'request_id': 'req-85d95b53-af8b-405a-a3db-74fef617f95a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:01.347250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e93d7ae01c94657ab3a2ad37a8ceb7b', 'request_id': 'req-85d95b53-af8b-405a-a3db-74fef617f95a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:01.355219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e93d7ae01c94657ab3a2ad37a8ceb7b', 'request_id': 'req-85d95b53-af8b-405a-a3db-74fef617f95a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:01.368771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] The token cache is being invalidate because project 9e93d7ae01c94657ab3a2ad37a8ceb7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:06:01.369135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:01.369956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85d95b53-af8b-405a-a3db-74fef617f95a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:01.374998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1372/2749] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:06:01 2022] DELETE /identity/v3/projects/9e93d7ae01c94657ab3a2ad37a8ceb7b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:06:01.380171 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661a167-a8c0-4e90-8b50-e546a148b757 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.380655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661a167-a8c0-4e90-8b50-e546a148b757 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.381044 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5661a167-a8c0-4e90-8b50-e546a148b757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.412519 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5661a167-a8c0-4e90-8b50-e546a148b757 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:06:01.413326 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5661a167-a8c0-4e90-8b50-e546a148b757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:01.490477 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1378/2750] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:06:01.497036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228f96a-5366-47ff-a622-9bf57d3887c1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.497401 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228f96a-5366-47ff-a622-9bf57d3887c1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.497751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228f96a-5366-47ff-a622-9bf57d3887c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.523798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0228f96a-5366-47ff-a622-9bf57d3887c1 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:06:01.524678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0228f96a-5366-47ff-a622-9bf57d3887c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:01.537742 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1373/2751] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:06:01.539109 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661a167-a8c0-4e90-8b50-e546a148b757 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:01.549812 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5661a167-a8c0-4e90-8b50-e546a148b757 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:01.554805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0d833-50c0-40af-9caa-140c33a6c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:01.555432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0d833-50c0-40af-9caa-140c33a6c633 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.555711 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0d833-50c0-40af-9caa-140c33a6c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.555983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0d833-50c0-40af-9caa-140c33a6c633 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.556683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0d833-50c0-40af-9caa-140c33a6c633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:06:01.556989 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0d833-50c0-40af-9caa-140c33a6c633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:01.558638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0d833-50c0-40af-9caa-140c33a6c633 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:01.575837 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1379/2752] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:06:01.581059 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228f96a-5366-47ff-a622-9bf57d3887c1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:01.590488 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228f96a-5366-47ff-a622-9bf57d3887c1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:01.595794 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f567fc-75f7-4b0c-84bb-7b486621f0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:01.596516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f567fc-75f7-4b0c-84bb-7b486621f0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.596857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f567fc-75f7-4b0c-84bb-7b486621f0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.597219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f567fc-75f7-4b0c-84bb-7b486621f0a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.597885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f567fc-75f7-4b0c-84bb-7b486621f0a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:01.599867 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f567fc-75f7-4b0c-84bb-7b486621f0a4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:01.643179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1374/2753] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:06:01 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:06:01.643539 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0d833-50c0-40af-9caa-140c33a6c633 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:01.643884 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:01.648310 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0d833-50c0-40af-9caa-140c33a6c633 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:01.710161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6270bf-5bdd-44ee-8a1c-5f5fe6247513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:01.710813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6270bf-5bdd-44ee-8a1c-5f5fe6247513 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.711093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6270bf-5bdd-44ee-8a1c-5f5fe6247513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.711361 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6270bf-5bdd-44ee-8a1c-5f5fe6247513 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.712054 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6270bf-5bdd-44ee-8a1c-5f5fe6247513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:01.713973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6270bf-5bdd-44ee-8a1c-5f5fe6247513 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:01.736358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1380/2754] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:06:01 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:06:01.740146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f567fc-75f7-4b0c-84bb-7b486621f0a4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:01.750653 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f567fc-75f7-4b0c-84bb-7b486621f0a4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:01.756979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce4ebe-d1c1-4994-86b1-de42128f04dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:01.757660 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce4ebe-d1c1-4994-86b1-de42128f04dc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.758001 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce4ebe-d1c1-4994-86b1-de42128f04dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.758341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce4ebe-d1c1-4994-86b1-de42128f04dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.759135 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce4ebe-d1c1-4994-86b1-de42128f04dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:01.760827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce4ebe-d1c1-4994-86b1-de42128f04dc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:01.774927 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1375/2755] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:06:01.778754 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6270bf-5bdd-44ee-8a1c-5f5fe6247513 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:01.789129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6270bf-5bdd-44ee-8a1c-5f5fe6247513 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:01.793924 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf03460-d07a-43d1-9503-e59078f16eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:01.794871 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf03460-d07a-43d1-9503-e59078f16eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.795119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf03460-d07a-43d1-9503-e59078f16eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.795438 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf03460-d07a-43d1-9503-e59078f16eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.796136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf03460-d07a-43d1-9503-e59078f16eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:01.797732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf03460-d07a-43d1-9503-e59078f16eb2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:01.810388 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1381/2756] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:06:01.815355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce4ebe-d1c1-4994-86b1-de42128f04dc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:01.826649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce4ebe-d1c1-4994-86b1-de42128f04dc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:01.831568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbd4d8-41f3-45aa-8890-3e0bc429ec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:01.832241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbd4d8-41f3-45aa-8890-3e0bc429ec9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.832522 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbd4d8-41f3-45aa-8890-3e0bc429ec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.832798 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbd4d8-41f3-45aa-8890-3e0bc429ec9b admin admin] PATH_INFO: `/v3/projects/5b1d1ed7bf5d4ebda853b5684bff2bae/users/f6bf1995ced14c6f83cb6305c15c2511/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.848896 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbd4d8-41f3-45aa-8890-3e0bc429ec9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1d1ed7bf5d4ebda853b5684bff2bae, user_id=f6bf1995ced14c6f83cb6305c15c2511, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:01.851735 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbd4d8-41f3-45aa-8890-3e0bc429ec9b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:01.870626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1376/2757] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:06:01 2022] PUT /identity/v3/projects/5b1d1ed7bf5d4ebda853b5684bff2bae/users/f6bf1995ced14c6f83cb6305c15c2511/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:06:01.881875 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1fe89-3dbc-49b9-8111-2fb15c772979 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:01.882161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1fe89-3dbc-49b9-8111-2fb15c772979 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:01.882442 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1fe89-3dbc-49b9-8111-2fb15c772979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:01.911489 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-19f1fe89-3dbc-49b9-8111-2fb15c772979 None None] MFA Rules not processed for user `f6bf1995ced14c6f83cb6305c15c2511`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:06:01.912716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-19f1fe89-3dbc-49b9-8111-2fb15c772979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:01.978525 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1382/2758] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:06:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:06:01.990066 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbd4d8-41f3-45aa-8890-3e0bc429ec9b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:01.991847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73bbd4d8-41f3-45aa-8890-3e0bc429ec9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:02.002783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:02.080143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbd4d8-41f3-45aa-8890-3e0bc429ec9b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:02.084128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc093e00-a5eb-44f9-8a05-f35633c05cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:02.084765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc093e00-a5eb-44f9-8a05-f35633c05cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:02.085052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc093e00-a5eb-44f9-8a05-f35633c05cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:02.085332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc093e00-a5eb-44f9-8a05-f35633c05cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:02.100260 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc093e00-a5eb-44f9-8a05-f35633c05cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:02.101770 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc093e00-a5eb-44f9-8a05-f35633c05cf0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:02.114006 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1377/2759] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:06:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:06:03.837099 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1fe89-3dbc-49b9-8111-2fb15c772979 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:03.838752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-19f1fe89-3dbc-49b9-8111-2fb15c772979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:03.919536 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1fe89-3dbc-49b9-8111-2fb15c772979 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:03.924833 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:03.925478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:03.925761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:03.926032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] PATH_INFO: `/v3/users/42d3708ddfcc491cb366054dcd5f8881` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:03.927244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d3708ddfcc491cb366054dcd5f8881)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:03.929170 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:03.944123 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:03.980128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d3708ddfcc491cb366054dcd5f8881'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:03.987820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d3708ddfcc491cb366054dcd5f8881'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:03.988486 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d3708ddfcc491cb366054dcd5f8881'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:03.989115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d3708ddfcc491cb366054dcd5f8881'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.001129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d3708ddfcc491cb366054dcd5f8881'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.021499 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1383/2760] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:06:03 2022] DELETE /identity/v3/users/42d3708ddfcc491cb366054dcd5f8881 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:06:04.231234 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc093e00-a5eb-44f9-8a05-f35633c05cf0 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:04.244631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc093e00-a5eb-44f9-8a05-f35633c05cf0 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:04.303566 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:04.304209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:04.304494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:04.304769 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] PATH_INFO: `/v3/projects/abbb209e2c194596bb3e8b9a6880646e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:04.305940 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=abbb209e2c194596bb3e8b9a6880646e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:04.307884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:04.353792 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abbb209e2c194596bb3e8b9a6880646e', 'request_id': 'req-671e08c1-7653-4afc-833c-95710f701ccd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.361806 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abbb209e2c194596bb3e8b9a6880646e', 'request_id': 'req-671e08c1-7653-4afc-833c-95710f701ccd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.367438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abbb209e2c194596bb3e8b9a6880646e', 'request_id': 'req-671e08c1-7653-4afc-833c-95710f701ccd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.378855 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] The token cache is being invalidate because project abbb209e2c194596bb3e8b9a6880646e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:06:04.379145 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.379780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.381326 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1378/2761] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:06:04 2022] DELETE /identity/v3/projects/abbb209e2c194596bb3e8b9a6880646e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:06:04.385336 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:04.386777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:04.400302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89db015-93f6-4046-80e7-96d49807c224 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:04.457735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:04.458418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:04.458721 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:04.458994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] PATH_INFO: `/v3/users/3b52445a83c44ed8ade18cfe23cdac14` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:04.460206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b52445a83c44ed8ade18cfe23cdac14)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:04.462160 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:04.509702 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b52445a83c44ed8ade18cfe23cdac14'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.516342 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b52445a83c44ed8ade18cfe23cdac14'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.517779 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b52445a83c44ed8ade18cfe23cdac14'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.518380 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b52445a83c44ed8ade18cfe23cdac14'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.530161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b52445a83c44ed8ade18cfe23cdac14'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.543014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1384/2762] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:06:04 2022] DELETE /identity/v3/users/3b52445a83c44ed8ade18cfe23cdac14 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:06:04.724136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:04.725662 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:04.739574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e08c1-7653-4afc-833c-95710f701ccd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:04.779636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:04.800612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:04.801252 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:04.801529 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:04.801801 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] PATH_INFO: `/v3/projects/a1b74e0af6924eaca6c3e94efb720a76` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:04.802979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b74e0af6924eaca6c3e94efb720a76)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:04.804874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:04.852675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b74e0af6924eaca6c3e94efb720a76', 'request_id': 'req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.862700 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b74e0af6924eaca6c3e94efb720a76', 'request_id': 'req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.868656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b74e0af6924eaca6c3e94efb720a76', 'request_id': 'req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.881179 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] The token cache is being invalidate because project a1b74e0af6924eaca6c3e94efb720a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:06:04.881464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.882115 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c4609d4-f2ef-4cbc-8b3e-4136efd0b846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:04.883664 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1379/2763] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:06:04 2022] DELETE /identity/v3/projects/a1b74e0af6924eaca6c3e94efb720a76 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:06:09.670714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:09.672138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:09.685562 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581ecd0-a846-42a2-8ce0-948de3fb2122 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:09.693362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:09.743106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:09.743778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:09.744052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:09.744321 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] PATH_INFO: `/v3/users/f6bf1995ced14c6f83cb6305c15c2511` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:09.745598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6bf1995ced14c6f83cb6305c15c2511)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:09.747594 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:09.792145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6bf1995ced14c6f83cb6305c15c2511'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:09.800030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6bf1995ced14c6f83cb6305c15c2511'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:09.800799 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6bf1995ced14c6f83cb6305c15c2511'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:09.801389 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6bf1995ced14c6f83cb6305c15c2511'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:09.813306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6bf1995ced14c6f83cb6305c15c2511'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:09.825777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1385/2764] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:06:09 2022] DELETE /identity/v3/users/f6bf1995ced14c6f83cb6305c15c2511 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:06:09.830245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bbea8-ed1b-44a0-aa05-c35c9a24d733 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:09.830757 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bbea8-ed1b-44a0-aa05-c35c9a24d733 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:09.830857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bbea8-ed1b-44a0-aa05-c35c9a24d733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:09.850392 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1d2bbea8-ed1b-44a0-aa05-c35c9a24d733 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:06:09.851616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d2bbea8-ed1b-44a0-aa05-c35c9a24d733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:09.912187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1380/2765] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:06:09.922038 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:09.923450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:09.989644 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957112c4-9376-47ee-95ae-c0a8d1f30f3c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:09.997373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c40368-fcb2-41a8-91bb-74dbe308da65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:09.998161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c40368-fcb2-41a8-91bb-74dbe308da65 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:09.998470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c40368-fcb2-41a8-91bb-74dbe308da65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:09.998734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c40368-fcb2-41a8-91bb-74dbe308da65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:10.016657 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c40368-fcb2-41a8-91bb-74dbe308da65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:10.019086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c40368-fcb2-41a8-91bb-74dbe308da65 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:10.030622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1386/2766] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:06:10.619433 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bbea8-ed1b-44a0-aa05-c35c9a24d733 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:10.621012 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d2bbea8-ed1b-44a0-aa05-c35c9a24d733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:10.709180 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bbea8-ed1b-44a0-aa05-c35c9a24d733 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:10.713848 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:10.714531 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:10.714816 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:10.715092 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] PATH_INFO: `/v3/projects/5b1d1ed7bf5d4ebda853b5684bff2bae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:10.716269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b1d1ed7bf5d4ebda853b5684bff2bae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:10.718182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:10.720472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:10.766874 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b1d1ed7bf5d4ebda853b5684bff2bae', 'request_id': 'req-a66d2690-992d-497f-9c69-1fde2cc60403'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:10.774128 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b1d1ed7bf5d4ebda853b5684bff2bae', 'request_id': 'req-a66d2690-992d-497f-9c69-1fde2cc60403'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:10.779695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b1d1ed7bf5d4ebda853b5684bff2bae', 'request_id': 'req-a66d2690-992d-497f-9c69-1fde2cc60403'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:10.790843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] The token cache is being invalidate because project 5b1d1ed7bf5d4ebda853b5684bff2bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:06:10.791143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:10.791751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a66d2690-992d-497f-9c69-1fde2cc60403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:06:10.793244 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1381/2767] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:06:10 2022] DELETE /identity/v3/projects/5b1d1ed7bf5d4ebda853b5684bff2bae => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:06:20.025264 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=74,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:20.787877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=23,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:48.060682 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c40368-fcb2-41a8-91bb-74dbe308da65 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:48.062061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e0c40368-fcb2-41a8-91bb-74dbe308da65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:48.156269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c40368-fcb2-41a8-91bb-74dbe308da65 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:48.204424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:48.205031 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:48.205307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:48.205576 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:48.207425 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:48.265829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:48.267305 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:48.276513 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1387/2768] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:06:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:06:55.142738 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc6d37-4a38-4c3e-905b-acd607663a32 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:55.143991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc6d37-4a38-4c3e-905b-acd607663a32 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:55.144105 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc6d37-4a38-4c3e-905b-acd607663a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:55.164109 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6adc6d37-4a38-4c3e-905b-acd607663a32 None None] MFA Rules not processed for user `407ad282edd74fb1adf62fb63306779c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:06:55.165305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6adc6d37-4a38-4c3e-905b-acd607663a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:55.172341 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1382/2769] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:06:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:06:56.353774 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:06:56.355392 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:56.390281 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:06:56.446928 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fda0b-813b-43d6-b027-3ddda4a3a610 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:06:56.451888 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:06:56.452493 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:06:56.452790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:06:56.453059 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:06:56.454849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:06:56.466601 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:06:56.468151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:06:56.478655 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1388/2770] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:06:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:07:05.159231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:07:06.474603 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=27 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:07:13.758114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc6d37-4a38-4c3e-905b-acd607663a32 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:07:13.759649 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6adc6d37-4a38-4c3e-905b-acd607663a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:07:13.840672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc6d37-4a38-4c3e-905b-acd607663a32 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:07:13.844337 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d76ad-10d7-41fa-9758-687ab3c6a637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:07:13.845576 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d76ad-10d7-41fa-9758-687ab3c6a637 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:07:13.846069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d76ad-10d7-41fa-9758-687ab3c6a637 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:07:13.846460 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d76ad-10d7-41fa-9758-687ab3c6a637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:07:13.848328 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-829d76ad-10d7-41fa-9758-687ab3c6a637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:07:13.915034 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d76ad-10d7-41fa-9758-687ab3c6a637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:07:13.916467 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d76ad-10d7-41fa-9758-687ab3c6a637 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:07:13.926548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1383/2771] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:07:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:07:23.921944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=59 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:08:39.120307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:08:39.121788 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:39.134031 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1ba83-53ae-4935-ab36-57df86bedb71 service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:08:39.137613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f283d-2bea-41f6-8977-aecf08d17073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:08:39.138209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f283d-2bea-41f6-8977-aecf08d17073 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:08:39.138512 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f283d-2bea-41f6-8977-aecf08d17073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:08:39.138783 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f283d-2bea-41f6-8977-aecf08d17073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:08:39.140447 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-490f283d-2bea-41f6-8977-aecf08d17073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:39.228150 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f283d-2bea-41f6-8977-aecf08d17073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:08:39.229590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f283d-2bea-41f6-8977-aecf08d17073 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:08:39.250134 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1389/2772] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:08:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:08:40.426042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d76ad-10d7-41fa-9758-687ab3c6a637 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:08:40.434934 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d76ad-10d7-41fa-9758-687ab3c6a637 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:08:40.438744 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff96a74-af4f-4c52-9867-4d7f3f8b3597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:08:40.439355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff96a74-af4f-4c52-9867-4d7f3f8b3597 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:08:40.439639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff96a74-af4f-4c52-9867-4d7f3f8b3597 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:08:40.439918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff96a74-af4f-4c52-9867-4d7f3f8b3597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:08:40.441706 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ff96a74-af4f-4c52-9867-4d7f3f8b3597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:40.455720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff96a74-af4f-4c52-9867-4d7f3f8b3597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:08:40.457146 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff96a74-af4f-4c52-9867-4d7f3f8b3597 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:08:40.468124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1384/2773] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:08:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:08:44.900034 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4dda4-5098-40b3-9db6-fada6c32593e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:08:44.900572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4dda4-5098-40b3-9db6-fada6c32593e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:08:44.900572 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4dda4-5098-40b3-9db6-fada6c32593e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:08:44.921229 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e4e4dda4-5098-40b3-9db6-fada6c32593e None None] MFA Rules not processed for user `3a805f5dec66484c8d950b6bc52e3e03`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:08:44.922503 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e4e4dda4-5098-40b3-9db6-fada6c32593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:44.980634 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1390/2774] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:08:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:08:45.290045 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff96a74-af4f-4c52-9867-4d7f3f8b3597 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:08:45.299518 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff96a74-af4f-4c52-9867-4d7f3f8b3597 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:08:45.303211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09974519-ea62-4c3a-96f0-85c2b81b6c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:08:45.303902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09974519-ea62-4c3a-96f0-85c2b81b6c10 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:08:45.304250 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09974519-ea62-4c3a-96f0-85c2b81b6c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:08:45.304589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09974519-ea62-4c3a-96f0-85c2b81b6c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:08:45.306431 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-09974519-ea62-4c3a-96f0-85c2b81b6c10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:45.316292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09974519-ea62-4c3a-96f0-85c2b81b6c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:08:45.317778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09974519-ea62-4c3a-96f0-85c2b81b6c10 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:08:45.326022 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1385/2775] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:08:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:08:45.659437 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4dda4-5098-40b3-9db6-fada6c32593e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:08:45.671634 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4dda4-5098-40b3-9db6-fada6c32593e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:08:45.677365 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:08:45.678012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:08:45.678293 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:08:45.678596 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:08:45.680364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:45.692660 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:08:45.694276 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:08:45.705202 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1391/2776] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:08:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:08:49.508736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09974519-ea62-4c3a-96f0-85c2b81b6c10 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:08:49.518667 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09974519-ea62-4c3a-96f0-85c2b81b6c10 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:08:49.522431 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7d4d5-30f8-4ec9-8c80-ddc0e9b48ed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:08:49.523069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7d4d5-30f8-4ec9-8c80-ddc0e9b48ed6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:08:49.523345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7d4d5-30f8-4ec9-8c80-ddc0e9b48ed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:08:49.523616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7d4d5-30f8-4ec9-8c80-ddc0e9b48ed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:08:49.525441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ac7d4d5-30f8-4ec9-8c80-ddc0e9b48ed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:49.538399 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7d4d5-30f8-4ec9-8c80-ddc0e9b48ed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:08:49.539857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7d4d5-30f8-4ec9-8c80-ddc0e9b48ed6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:08:49.548157 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1386/2777] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:08:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:08:52.705656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:08:52.707114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:52.726185 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28789f8c-9ff2-4200-8497-2744a682b67d service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:08:52.732479 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1dc3ad467d9496cb3e0086a0cd8215e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:08:52.733032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:08:52.733315 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:08:52.733615 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:08:52.735533 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:52.748573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:08:52.751121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:08:52.762302 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1392/2778] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:08:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:08:56.196353 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7d4d5-30f8-4ec9-8c80-ddc0e9b48ed6 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:08:56.206799 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7d4d5-30f8-4ec9-8c80-ddc0e9b48ed6 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:08:56.210731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab482e1-20ba-4831-99e8-0d646641280d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:08:56.211345 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab482e1-20ba-4831-99e8-0d646641280d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:08:56.211627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab482e1-20ba-4831-99e8-0d646641280d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:08:56.211902 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab482e1-20ba-4831-99e8-0d646641280d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:08:56.213737 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cab482e1-20ba-4831-99e8-0d646641280d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:08:56.280079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab482e1-20ba-4831-99e8-0d646641280d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:08:56.282072 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab482e1-20ba-4831-99e8-0d646641280d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:08:56.292240 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1387/2779] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:09:02.757645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=83 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:09:02.995182 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:09:02.996732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:09:03.013338 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001b51a-2c21-4ca4-a35f-8681da0f5661 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:09:03.020041 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:09:03.020685 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:09:03.020960 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:09:03.021232 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] PATH_INFO: `/v3/users/407ad282edd74fb1adf62fb63306779c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:09:03.022488 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=407ad282edd74fb1adf62fb63306779c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:09:03.024422 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:09:03.070732 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '407ad282edd74fb1adf62fb63306779c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.077492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '407ad282edd74fb1adf62fb63306779c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.078237 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '407ad282edd74fb1adf62fb63306779c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.079056 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '407ad282edd74fb1adf62fb63306779c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.090756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '407ad282edd74fb1adf62fb63306779c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.103378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1393/2780] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:09:02 2022] DELETE /identity/v3/users/407ad282edd74fb1adf62fb63306779c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:09:03.276535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab482e1-20ba-4831-99e8-0d646641280d service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:09:03.287750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab482e1-20ba-4831-99e8-0d646641280d service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:09:03.346947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:09:03.347612 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:09:03.347891 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:09:03.348165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] PATH_INFO: `/v3/projects/db365a6a9f6041a7a8749fd1a1e3d488` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:09:03.349305 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db365a6a9f6041a7a8749fd1a1e3d488)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:09:03.351263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:09:03.398152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db365a6a9f6041a7a8749fd1a1e3d488', 'request_id': 'req-f1c71c55-efeb-443c-8eb3-961d361fdb59'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.405388 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db365a6a9f6041a7a8749fd1a1e3d488', 'request_id': 'req-f1c71c55-efeb-443c-8eb3-961d361fdb59'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.407172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:09:03.413125 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db365a6a9f6041a7a8749fd1a1e3d488', 'request_id': 'req-f1c71c55-efeb-443c-8eb3-961d361fdb59'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.425172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] The token cache is being invalidate because project db365a6a9f6041a7a8749fd1a1e3d488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:09:03.425457 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.426131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.427642 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1388/2781] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:09:03 2022] DELETE /identity/v3/projects/db365a6a9f6041a7a8749fd1a1e3d488 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:09:03.431675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:09:03.433061 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:09:03.446559 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2463e92-fa80-4ee1-a795-c536799f60ff admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:09:03.503550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:09:03.504181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:09:03.504455 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:09:03.504725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] PATH_INFO: `/v3/users/7c996bb1073f43fc9c7ce5509f2c4cc1` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:09:03.505994 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c996bb1073f43fc9c7ce5509f2c4cc1)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:09:03.508012 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:09:03.554373 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c996bb1073f43fc9c7ce5509f2c4cc1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.561585 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c996bb1073f43fc9c7ce5509f2c4cc1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.562379 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c996bb1073f43fc9c7ce5509f2c4cc1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.563018 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c996bb1073f43fc9c7ce5509f2c4cc1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.574683 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c996bb1073f43fc9c7ce5509f2c4cc1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.576324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:09:03.589523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1394/2782] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:09:03 2022] DELETE /identity/v3/users/7c996bb1073f43fc9c7ce5509f2c4cc1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:09:03.756618 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:09:03.758013 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:09:03.772631 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c71c55-efeb-443c-8eb3-961d361fdb59 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:09:03.836784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:09:03.837441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:09:03.837716 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:09:03.837986 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] PATH_INFO: `/v3/projects/045ba324caf94e62a62b3ed245adf7b6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:09:03.839162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=045ba324caf94e62a62b3ed245adf7b6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:09:03.841120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:09:03.887477 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '045ba324caf94e62a62b3ed245adf7b6', 'request_id': 'req-a5775724-401c-4e47-b787-6a8fd9404204'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.897144 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '045ba324caf94e62a62b3ed245adf7b6', 'request_id': 'req-a5775724-401c-4e47-b787-6a8fd9404204'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.903185 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '045ba324caf94e62a62b3ed245adf7b6', 'request_id': 'req-a5775724-401c-4e47-b787-6a8fd9404204'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.914657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] The token cache is being invalidate because project 045ba324caf94e62a62b3ed245adf7b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:09:03.914947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.916116 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:03.917638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1389/2783] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:09:03 2022] DELETE /identity/v3/projects/045ba324caf94e62a62b3ed245adf7b6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:09:13.588862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:09:13.911616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=9,keystone:SELECT=56,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:09:59.672243 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:09:59.673761 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:09:59.739051 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:09:59.741819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:09:59.763868 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d86116-70db-47cf-9905-e7c10cbd1b4b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:09:59.821517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5775724-401c-4e47-b787-6a8fd9404204 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:09:59.826843 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:09:59.828387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:09:59.828750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:09:59.829100 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] PATH_INFO: `/v3/users/3a805f5dec66484c8d950b6bc52e3e03` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:09:59.830591 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a805f5dec66484c8d950b6bc52e3e03)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:09:59.832607 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:09:59.833804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c124716-db6a-427d-90a3-fb37f12bd157 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:09:59.834587 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c124716-db6a-427d-90a3-fb37f12bd157 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:09:59.834930 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c124716-db6a-427d-90a3-fb37f12bd157 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:09:59.835267 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c124716-db6a-427d-90a3-fb37f12bd157 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:09:59.837093 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4c124716-db6a-427d-90a3-fb37f12bd157 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:09:59.849726 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c124716-db6a-427d-90a3-fb37f12bd157 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:09:59.851362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c124716-db6a-427d-90a3-fb37f12bd157 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:09:59.862136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1395/2784] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:09:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:09:59.895221 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a805f5dec66484c8d950b6bc52e3e03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:59.906136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a805f5dec66484c8d950b6bc52e3e03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:59.907106 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a805f5dec66484c8d950b6bc52e3e03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:59.907656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a805f5dec66484c8d950b6bc52e3e03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:59.920103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-debd152d-4026-4f74-a29c-340d9c0dd7eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a805f5dec66484c8d950b6bc52e3e03'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:09:59.933279 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1390/2785] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:09:59 2022] DELETE /identity/v3/users/3a805f5dec66484c8d950b6bc52e3e03 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:00.105752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c124716-db6a-427d-90a3-fb37f12bd157 service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:00.116923 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c124716-db6a-427d-90a3-fb37f12bd157 service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:00.174863 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:00.175430 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.175708 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.175980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] PATH_INFO: `/v3/projects/d9f8c70dc5de488698a8ab775c618088` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.177129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f8c70dc5de488698a8ab775c618088)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:00.179051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:00.216055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:00.226807 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f8c70dc5de488698a8ab775c618088', 'request_id': 'req-664b22ef-9fb7-45e1-af45-a717dede2883'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:00.234496 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f8c70dc5de488698a8ab775c618088', 'request_id': 'req-664b22ef-9fb7-45e1-af45-a717dede2883'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:00.240005 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f8c70dc5de488698a8ab775c618088', 'request_id': 'req-664b22ef-9fb7-45e1-af45-a717dede2883'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:00.251030 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] The token cache is being invalidate because project d9f8c70dc5de488698a8ab775c618088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:00.251314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:00.251944 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-664b22ef-9fb7-45e1-af45-a717dede2883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:00.253428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1396/2786] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:00 2022] DELETE /identity/v3/projects/d9f8c70dc5de488698a8ab775c618088 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:00.258383 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504eba4-a7bb-4150-848d-ac50c54cdfcb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.258718 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504eba4-a7bb-4150-848d-ac50c54cdfcb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.258993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504eba4-a7bb-4150-848d-ac50c54cdfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.316756 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b504eba4-a7bb-4150-848d-ac50c54cdfcb None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:00.317906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b504eba4-a7bb-4150-848d-ac50c54cdfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:00.363162 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:00.380758 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1391/2787] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:00.388659 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4b194-5cb4-4abc-9bf1-e5a5a1f3d11b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.388943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4b194-5cb4-4abc-9bf1-e5a5a1f3d11b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.389218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4b194-5cb4-4abc-9bf1-e5a5a1f3d11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.408426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1ff4b194-5cb4-4abc-9bf1-e5a5a1f3d11b None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:00.409630 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1ff4b194-5cb4-4abc-9bf1-e5a5a1f3d11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:00.417967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1397/2788] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:00.422753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504eba4-a7bb-4150-848d-ac50c54cdfcb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:00.432499 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504eba4-a7bb-4150-848d-ac50c54cdfcb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:00.437982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37610f53-3e9d-44b0-a6fc-3537a813f55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:00.438629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37610f53-3e9d-44b0-a6fc-3537a813f55c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.438912 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37610f53-3e9d-44b0-a6fc-3537a813f55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.439186 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37610f53-3e9d-44b0-a6fc-3537a813f55c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.439878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37610f53-3e9d-44b0-a6fc-3537a813f55c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:10:00.440182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37610f53-3e9d-44b0-a6fc-3537a813f55c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:00.441777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37610f53-3e9d-44b0-a6fc-3537a813f55c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:00.454056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1392/2789] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:10:00.458604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4b194-5cb4-4abc-9bf1-e5a5a1f3d11b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:00.468341 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4b194-5cb4-4abc-9bf1-e5a5a1f3d11b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:00.473066 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae13c90-52fb-40ba-af4c-cb12e553bd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:00.473667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae13c90-52fb-40ba-af4c-cb12e553bd61 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.473943 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae13c90-52fb-40ba-af4c-cb12e553bd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.474211 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae13c90-52fb-40ba-af4c-cb12e553bd61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.474911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae13c90-52fb-40ba-af4c-cb12e553bd61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:00.476802 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae13c90-52fb-40ba-af4c-cb12e553bd61 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:00.513250 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1398/2790] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:00 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:00.517548 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37610f53-3e9d-44b0-a6fc-3537a813f55c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:00.527708 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37610f53-3e9d-44b0-a6fc-3537a813f55c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:00.583215 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1b9fb-35bd-4b06-9f7c-5930cb0a4320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:00.583750 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1b9fb-35bd-4b06-9f7c-5930cb0a4320 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.584027 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1b9fb-35bd-4b06-9f7c-5930cb0a4320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.584296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1b9fb-35bd-4b06-9f7c-5930cb0a4320 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.584944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1b9fb-35bd-4b06-9f7c-5930cb0a4320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:00.586889 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae1b9fb-35bd-4b06-9f7c-5930cb0a4320 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:00.609480 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1393/2791] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:00 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:00.613558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae13c90-52fb-40ba-af4c-cb12e553bd61 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:00.623898 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae13c90-52fb-40ba-af4c-cb12e553bd61 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:00.628460 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e70ef-7480-4aa2-8524-c43a9ab52ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:00.629057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e70ef-7480-4aa2-8524-c43a9ab52ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.629333 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e70ef-7480-4aa2-8524-c43a9ab52ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.629604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e70ef-7480-4aa2-8524-c43a9ab52ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.630284 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e70ef-7480-4aa2-8524-c43a9ab52ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:00.631906 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e70ef-7480-4aa2-8524-c43a9ab52ce2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:00.641627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1399/2792] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:00.645495 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1b9fb-35bd-4b06-9f7c-5930cb0a4320 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:00.655928 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1b9fb-35bd-4b06-9f7c-5930cb0a4320 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:00.660535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849ce0b-1ee7-49de-90ba-edeb24e6da9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:00.661134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849ce0b-1ee7-49de-90ba-edeb24e6da9c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.661414 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849ce0b-1ee7-49de-90ba-edeb24e6da9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.661688 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849ce0b-1ee7-49de-90ba-edeb24e6da9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.662405 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849ce0b-1ee7-49de-90ba-edeb24e6da9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:00.664270 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849ce0b-1ee7-49de-90ba-edeb24e6da9c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:00.675239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1394/2793] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:10:00.679227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e70ef-7480-4aa2-8524-c43a9ab52ce2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:00.689387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e70ef-7480-4aa2-8524-c43a9ab52ce2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:00.694046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1116c0-5401-43d2-a2e6-88d3ce5ba6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:00.694731 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1116c0-5401-43d2-a2e6-88d3ce5ba6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.695014 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1116c0-5401-43d2-a2e6-88d3ce5ba6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.695287 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1116c0-5401-43d2-a2e6-88d3ce5ba6bc admin admin] PATH_INFO: `/v3/projects/0830b6f529ff4f8fb4384019d76a5a7a/users/e283e8e0616646dcb83a2b3354818124/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.709778 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1116c0-5401-43d2-a2e6-88d3ce5ba6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0830b6f529ff4f8fb4384019d76a5a7a, user_id=e283e8e0616646dcb83a2b3354818124, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:00.712454 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1116c0-5401-43d2-a2e6-88d3ce5ba6bc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:00.726631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1400/2794] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:00 2022] PUT /identity/v3/projects/0830b6f529ff4f8fb4384019d76a5a7a/users/e283e8e0616646dcb83a2b3354818124/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:00.738622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bab4d-75a3-4505-9187-3effaced7f2a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.739011 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bab4d-75a3-4505-9187-3effaced7f2a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.739213 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bab4d-75a3-4505-9187-3effaced7f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.767358 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-192bab4d-75a3-4505-9187-3effaced7f2a None None] MFA Rules not processed for user `e283e8e0616646dcb83a2b3354818124`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:00.768589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-192bab4d-75a3-4505-9187-3effaced7f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:00.839714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1395/2795] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:00.844790 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1116c0-5401-43d2-a2e6-88d3ce5ba6bc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:00.907716 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1116c0-5401-43d2-a2e6-88d3ce5ba6bc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:00.912340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eafbe-ccda-464a-bdee-fe5a7d9082c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:00.912952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eafbe-ccda-464a-bdee-fe5a7d9082c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:00.913227 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eafbe-ccda-464a-bdee-fe5a7d9082c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:00.913497 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eafbe-ccda-464a-bdee-fe5a7d9082c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:00.914398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eafbe-ccda-464a-bdee-fe5a7d9082c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:00.916306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eafbe-ccda-464a-bdee-fe5a7d9082c2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:00.942367 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1401/2796] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:00 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:00.946536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bab4d-75a3-4505-9187-3effaced7f2a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:00.957842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bab4d-75a3-4505-9187-3effaced7f2a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:00.999209 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:01.016584 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777955b-b63e-4eeb-9d5c-697ef6e876b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:01.017211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777955b-b63e-4eeb-9d5c-697ef6e876b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:01.017494 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777955b-b63e-4eeb-9d5c-697ef6e876b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:01.017771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777955b-b63e-4eeb-9d5c-697ef6e876b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:01.018487 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777955b-b63e-4eeb-9d5c-697ef6e876b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:01.020412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777955b-b63e-4eeb-9d5c-697ef6e876b5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:01.041622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1396/2797] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:00 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:01.045432 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eafbe-ccda-464a-bdee-fe5a7d9082c2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:01.056151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eafbe-ccda-464a-bdee-fe5a7d9082c2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:01.060763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0ab45-2399-4547-a130-1bfc8f9d5186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:01.061848 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0ab45-2399-4547-a130-1bfc8f9d5186 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:01.062136 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0ab45-2399-4547-a130-1bfc8f9d5186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:01.062412 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0ab45-2399-4547-a130-1bfc8f9d5186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:01.063146 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0ab45-2399-4547-a130-1bfc8f9d5186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:01.064728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0ab45-2399-4547-a130-1bfc8f9d5186 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:01.075988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1402/2798] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:01.080456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777955b-b63e-4eeb-9d5c-697ef6e876b5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:01.090698 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777955b-b63e-4eeb-9d5c-697ef6e876b5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:01.096136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df9c07-58ac-453e-9ab0-1640316bd9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:01.096793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df9c07-58ac-453e-9ab0-1640316bd9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:01.097076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df9c07-58ac-453e-9ab0-1640316bd9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:01.097354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df9c07-58ac-453e-9ab0-1640316bd9ae admin admin] PATH_INFO: `/v3/projects/78943b23bc0944e1a0de628039534b92/users/1d0a5b1bb55a4fea9b1ec6ab7e0d845b/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:01.113558 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df9c07-58ac-453e-9ab0-1640316bd9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=78943b23bc0944e1a0de628039534b92, user_id=1d0a5b1bb55a4fea9b1ec6ab7e0d845b, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:01.116322 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df9c07-58ac-453e-9ab0-1640316bd9ae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:01.129520 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1397/2799] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:01 2022] PUT /identity/v3/projects/78943b23bc0944e1a0de628039534b92/users/1d0a5b1bb55a4fea9b1ec6ab7e0d845b/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:01.133452 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0ab45-2399-4547-a130-1bfc8f9d5186 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:01.143866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0ab45-2399-4547-a130-1bfc8f9d5186 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:01.163364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:01.200212 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ee2e1-4ad6-4522-a513-058d52342776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:01.200820 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ee2e1-4ad6-4522-a513-058d52342776 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:01.201104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ee2e1-4ad6-4522-a513-058d52342776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:01.201360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ee2e1-4ad6-4522-a513-058d52342776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:01.202698 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ee2e1-4ad6-4522-a513-058d52342776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:01.204457 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ee2e1-4ad6-4522-a513-058d52342776 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:01.213962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1403/2800] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:01.218359 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df9c07-58ac-453e-9ab0-1640316bd9ae admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:01.228692 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df9c07-58ac-453e-9ab0-1640316bd9ae admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:01.233418 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5465fdb-ab98-478e-8ea5-0c37bebbb0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:01.234182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5465fdb-ab98-478e-8ea5-0c37bebbb0d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:01.234498 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5465fdb-ab98-478e-8ea5-0c37bebbb0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:01.234780 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5465fdb-ab98-478e-8ea5-0c37bebbb0d1 admin admin] PATH_INFO: `/v3/projects/78943b23bc0944e1a0de628039534b92/users/1d0a5b1bb55a4fea9b1ec6ab7e0d845b/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:01.236765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5465fdb-ab98-478e-8ea5-0c37bebbb0d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78943b23bc0944e1a0de628039534b92, user_id=1d0a5b1bb55a4fea9b1ec6ab7e0d845b, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:01.239446 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5465fdb-ab98-478e-8ea5-0c37bebbb0d1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:01.250789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1398/2801] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:01 2022] PUT /identity/v3/projects/78943b23bc0944e1a0de628039534b92/users/1d0a5b1bb55a4fea9b1ec6ab7e0d845b/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:01.260932 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f02fc-cb8c-4d2f-8c12-6d9ad3cac66f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:01.261266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f02fc-cb8c-4d2f-8c12-6d9ad3cac66f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:01.261541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f02fc-cb8c-4d2f-8c12-6d9ad3cac66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:01.290048 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d01f02fc-cb8c-4d2f-8c12-6d9ad3cac66f None None] MFA Rules not processed for user `1d0a5b1bb55a4fea9b1ec6ab7e0d845b`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:01.291300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d01f02fc-cb8c-4d2f-8c12-6d9ad3cac66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:01.362846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1404/2802] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:01.374382 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5465fdb-ab98-478e-8ea5-0c37bebbb0d1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:01.376111 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5465fdb-ab98-478e-8ea5-0c37bebbb0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:01.434384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5465fdb-ab98-478e-8ea5-0c37bebbb0d1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:01.438372 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cfd54-f24e-4438-bd5a-ef50fe15c559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:01.439004 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cfd54-f24e-4438-bd5a-ef50fe15c559 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:01.439282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cfd54-f24e-4438-bd5a-ef50fe15c559 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:01.439555 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cfd54-f24e-4438-bd5a-ef50fe15c559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:01.448445 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cfd54-f24e-4438-bd5a-ef50fe15c559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:01.449892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cfd54-f24e-4438-bd5a-ef50fe15c559 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:01.459310 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1399/2803] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:01.536426 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f02fc-cb8c-4d2f-8c12-6d9ad3cac66f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:01.546364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f02fc-cb8c-4d2f-8c12-6d9ad3cac66f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:01.549992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1660e7-0821-4d4e-8da3-4ffc01edaac0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:01.550632 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1660e7-0821-4d4e-8da3-4ffc01edaac0 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:01.550922 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1660e7-0821-4d4e-8da3-4ffc01edaac0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:01.551196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1660e7-0821-4d4e-8da3-4ffc01edaac0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:01.607137 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1660e7-0821-4d4e-8da3-4ffc01edaac0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:01.609386 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1660e7-0821-4d4e-8da3-4ffc01edaac0 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:01.618252 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1405/2804] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:10.455282 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cfd54-f24e-4438-bd5a-ef50fe15c559 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:10.516398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cfd54-f24e-4438-bd5a-ef50fe15c559 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:10.521509 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:10.522143 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:10.522425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:10.522733 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] PATH_INFO: `/v3/users/e283e8e0616646dcb83a2b3354818124` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:10.523896 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e283e8e0616646dcb83a2b3354818124)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:10.525815 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:10.566492 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:10.572151 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e283e8e0616646dcb83a2b3354818124'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:10.580057 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e283e8e0616646dcb83a2b3354818124'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:10.580720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e283e8e0616646dcb83a2b3354818124'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:10.581330 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e283e8e0616646dcb83a2b3354818124'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:10.593052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e283e8e0616646dcb83a2b3354818124'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:10.605536 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1400/2805] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:10 2022] DELETE /identity/v3/users/e283e8e0616646dcb83a2b3354818124 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:10.610641 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1c01d-d2ed-416b-9c87-6f41c71593dc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:10.611151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1c01d-d2ed-416b-9c87-6f41c71593dc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:10.611151 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1c01d-d2ed-416b-9c87-6f41c71593dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:10.630467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-12b1c01d-d2ed-416b-9c87-6f41c71593dc None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:10.631658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-12b1c01d-d2ed-416b-9c87-6f41c71593dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:10.669858 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:10.689669 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1406/2806] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:10.699395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:10.700779 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:10.758490 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc76b85-0041-4b33-84fa-ec16f7286191 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:10.762056 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8e0b7-e966-4a59-96f5-7eb7b57ef6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:10.762720 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8e0b7-e966-4a59-96f5-7eb7b57ef6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:10.762998 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8e0b7-e966-4a59-96f5-7eb7b57ef6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:10.763269 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8e0b7-e966-4a59-96f5-7eb7b57ef6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:10.772412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8e0b7-e966-4a59-96f5-7eb7b57ef6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:10.773836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8e0b7-e966-4a59-96f5-7eb7b57ef6b8 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:10.783130 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1401/2807] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:11.050256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1c01d-d2ed-416b-9c87-6f41c71593dc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:11.051713 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-12b1c01d-d2ed-416b-9c87-6f41c71593dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:11.064334 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1c01d-d2ed-416b-9c87-6f41c71593dc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:11.069174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:11.069865 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:11.070307 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:11.070604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] PATH_INFO: `/v3/projects/0830b6f529ff4f8fb4384019d76a5a7a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:11.071808 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0830b6f529ff4f8fb4384019d76a5a7a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:11.073764 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:11.121303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0830b6f529ff4f8fb4384019d76a5a7a', 'request_id': 'req-afa2212f-da3d-407c-8bef-b5265716665e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.128534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0830b6f529ff4f8fb4384019d76a5a7a', 'request_id': 'req-afa2212f-da3d-407c-8bef-b5265716665e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.134210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0830b6f529ff4f8fb4384019d76a5a7a', 'request_id': 'req-afa2212f-da3d-407c-8bef-b5265716665e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.145304 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] The token cache is being invalidate because project 0830b6f529ff4f8fb4384019d76a5a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:11.145586 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.146206 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.147784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1407/2808] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:11 2022] DELETE /identity/v3/projects/0830b6f529ff4f8fb4384019d76a5a7a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:11.151878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8e0b7-e966-4a59-96f5-7eb7b57ef6b8 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:11.161971 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8e0b7-e966-4a59-96f5-7eb7b57ef6b8 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:11.217537 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:11.218175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:11.218479 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:11.218760 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] PATH_INFO: `/v3/users/1d0a5b1bb55a4fea9b1ec6ab7e0d845b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:11.220440 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0a5b1bb55a4fea9b1ec6ab7e0d845b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:11.222598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:11.268857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0a5b1bb55a4fea9b1ec6ab7e0d845b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.275882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0a5b1bb55a4fea9b1ec6ab7e0d845b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.276568 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0a5b1bb55a4fea9b1ec6ab7e0d845b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.277172 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0a5b1bb55a4fea9b1ec6ab7e0d845b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.283484 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:11.292563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f7b192f-fb18-44a3-a8f2-a6459a4c1869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0a5b1bb55a4fea9b1ec6ab7e0d845b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.306274 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1402/2809] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:11 2022] DELETE /identity/v3/users/1d0a5b1bb55a4fea9b1ec6ab7e0d845b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:11.575343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:11.576787 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:11.591409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2212f-da3d-407c-8bef-b5265716665e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:11.648952 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:11.649605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:11.649846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:11.650114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] PATH_INFO: `/v3/projects/78943b23bc0944e1a0de628039534b92` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:11.651280 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78943b23bc0944e1a0de628039534b92)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:11.653158 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:11.697796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78943b23bc0944e1a0de628039534b92', 'request_id': 'req-db79f29c-d91a-4f8b-b882-4550615be4c1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.707785 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78943b23bc0944e1a0de628039534b92', 'request_id': 'req-db79f29c-d91a-4f8b-b882-4550615be4c1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.713255 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78943b23bc0944e1a0de628039534b92', 'request_id': 'req-db79f29c-d91a-4f8b-b882-4550615be4c1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.720995 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:11.726577 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] The token cache is being invalidate because project 78943b23bc0944e1a0de628039534b92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:11.726927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.727866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-db79f29c-d91a-4f8b-b882-4550615be4c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:11.730025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1408/2810] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:11 2022] DELETE /identity/v3/projects/78943b23bc0944e1a0de628039534b92 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:11.735884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db7fca-3270-4c65-ac9e-bd02101054d6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:11.736395 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db7fca-3270-4c65-ac9e-bd02101054d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:11.736542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db7fca-3270-4c65-ac9e-bd02101054d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:11.755831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d7db7fca-3270-4c65-ac9e-bd02101054d6 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:11.757026 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7db7fca-3270-4c65-ac9e-bd02101054d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:11.815355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1403/2811] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:11.824598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce651c0-d4c0-4595-adf5-b77c83523a8e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:11.824882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce651c0-d4c0-4595-adf5-b77c83523a8e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:11.825154 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce651c0-d4c0-4595-adf5-b77c83523a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:11.845026 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8ce651c0-d4c0-4595-adf5-b77c83523a8e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:11.846226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8ce651c0-d4c0-4595-adf5-b77c83523a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:11.854045 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1409/2812] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:11.858375 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db7fca-3270-4c65-ac9e-bd02101054d6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:11.868950 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db7fca-3270-4c65-ac9e-bd02101054d6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:11.873985 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c654ba-7d0e-4ccf-a09a-2713f33108b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:11.874847 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c654ba-7d0e-4ccf-a09a-2713f33108b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:11.875242 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c654ba-7d0e-4ccf-a09a-2713f33108b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:11.875598 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c654ba-7d0e-4ccf-a09a-2713f33108b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:11.876296 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c654ba-7d0e-4ccf-a09a-2713f33108b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:10:11.876599 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c654ba-7d0e-4ccf-a09a-2713f33108b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:11.878245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c654ba-7d0e-4ccf-a09a-2713f33108b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:11.889818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1404/2813] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:10:11.894376 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce651c0-d4c0-4595-adf5-b77c83523a8e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:11.903605 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce651c0-d4c0-4595-adf5-b77c83523a8e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:11.909846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fcba3-4df6-4a24-bdb8-9e3256a719c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:11.910483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fcba3-4df6-4a24-bdb8-9e3256a719c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:11.910794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fcba3-4df6-4a24-bdb8-9e3256a719c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:11.911085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fcba3-4df6-4a24-bdb8-9e3256a719c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:11.911752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fcba3-4df6-4a24-bdb8-9e3256a719c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:11.913882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fcba3-4df6-4a24-bdb8-9e3256a719c8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:11.940768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1410/2814] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:11 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:10:11.944859 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c654ba-7d0e-4ccf-a09a-2713f33108b0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:11.955060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c654ba-7d0e-4ccf-a09a-2713f33108b0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.012453 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec0e77-7003-4837-b98b-eb741b87eebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.013069 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec0e77-7003-4837-b98b-eb741b87eebb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.013354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec0e77-7003-4837-b98b-eb741b87eebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.013629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec0e77-7003-4837-b98b-eb741b87eebb admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.014303 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec0e77-7003-4837-b98b-eb741b87eebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.016812 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec0e77-7003-4837-b98b-eb741b87eebb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.037793 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1405/2815] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:11 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:12.042218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fcba3-4df6-4a24-bdb8-9e3256a719c8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.052946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fcba3-4df6-4a24-bdb8-9e3256a719c8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.057554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7d78e-e03d-4576-9996-0526d58ca87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.058153 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7d78e-e03d-4576-9996-0526d58ca87f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.058427 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7d78e-e03d-4576-9996-0526d58ca87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.058725 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7d78e-e03d-4576-9996-0526d58ca87f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.059408 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7d78e-e03d-4576-9996-0526d58ca87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.060987 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7d78e-e03d-4576-9996-0526d58ca87f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.071062 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1411/2816] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:12.074777 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec0e77-7003-4837-b98b-eb741b87eebb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.085377 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec0e77-7003-4837-b98b-eb741b87eebb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.089984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5c886-982a-4664-a719-eb4334b34fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.090624 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5c886-982a-4664-a719-eb4334b34fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.090908 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5c886-982a-4664-a719-eb4334b34fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.091182 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5c886-982a-4664-a719-eb4334b34fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.091876 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5c886-982a-4664-a719-eb4334b34fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.093464 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5c886-982a-4664-a719-eb4334b34fe2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.104644 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1406/2817] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:12.108717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7d78e-e03d-4576-9996-0526d58ca87f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.118667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7d78e-e03d-4576-9996-0526d58ca87f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.123476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf9bc8-a8d1-4b34-84b8-7b6c3ef422b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.124055 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf9bc8-a8d1-4b34-84b8-7b6c3ef422b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.124342 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf9bc8-a8d1-4b34-84b8-7b6c3ef422b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.124617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf9bc8-a8d1-4b34-84b8-7b6c3ef422b0 admin admin] PATH_INFO: `/v3/projects/407a3aaa2c404cbd8a65ad1feb3cd682/users/fe1a95e1d3ad4b6f87657b9d5c0e7e02/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.140051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf9bc8-a8d1-4b34-84b8-7b6c3ef422b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=407a3aaa2c404cbd8a65ad1feb3cd682, user_id=fe1a95e1d3ad4b6f87657b9d5c0e7e02, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.143450 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf9bc8-a8d1-4b34-84b8-7b6c3ef422b0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.155816 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1412/2818] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:12 2022] PUT /identity/v3/projects/407a3aaa2c404cbd8a65ad1feb3cd682/users/fe1a95e1d3ad4b6f87657b9d5c0e7e02/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:12.167036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfdff9-ff82-4d57-aa44-a7d0a423bff7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.167323 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfdff9-ff82-4d57-aa44-a7d0a423bff7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.167669 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfdff9-ff82-4d57-aa44-a7d0a423bff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.196249 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1bbfdff9-ff82-4d57-aa44-a7d0a423bff7 None None] MFA Rules not processed for user `fe1a95e1d3ad4b6f87657b9d5c0e7e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:12.197466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1bbfdff9-ff82-4d57-aa44-a7d0a423bff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:12.234258 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:12.267387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1407/2819] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:12.272811 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf9bc8-a8d1-4b34-84b8-7b6c3ef422b0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.341358 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf9bc8-a8d1-4b34-84b8-7b6c3ef422b0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.347119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf428f03-cc11-410d-a3f4-d48b44361211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.347730 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf428f03-cc11-410d-a3f4-d48b44361211 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.348009 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf428f03-cc11-410d-a3f4-d48b44361211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.348282 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf428f03-cc11-410d-a3f4-d48b44361211 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.348946 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf428f03-cc11-410d-a3f4-d48b44361211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.350867 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf428f03-cc11-410d-a3f4-d48b44361211 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.377940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1413/2820] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:12 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:12.381974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfdff9-ff82-4d57-aa44-a7d0a423bff7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.392931 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfdff9-ff82-4d57-aa44-a7d0a423bff7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.449721 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856a287-e49d-439f-b124-d581e47e0107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.450363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856a287-e49d-439f-b124-d581e47e0107 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.450675 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856a287-e49d-439f-b124-d581e47e0107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.450953 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856a287-e49d-439f-b124-d581e47e0107 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.451611 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856a287-e49d-439f-b124-d581e47e0107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.453651 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856a287-e49d-439f-b124-d581e47e0107 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.475350 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1408/2821] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:12 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:12.479346 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf428f03-cc11-410d-a3f4-d48b44361211 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.490042 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf428f03-cc11-410d-a3f4-d48b44361211 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.494613 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70818f-dc21-4671-bb2f-af168686ce13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.495210 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70818f-dc21-4671-bb2f-af168686ce13 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.495484 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70818f-dc21-4671-bb2f-af168686ce13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.495752 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70818f-dc21-4671-bb2f-af168686ce13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.496440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70818f-dc21-4671-bb2f-af168686ce13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.498107 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70818f-dc21-4671-bb2f-af168686ce13 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.508772 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1414/2822] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:12.512782 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856a287-e49d-439f-b124-d581e47e0107 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.522909 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856a287-e49d-439f-b124-d581e47e0107 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.527523 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4009390-688e-4e1a-aea0-3c740fce0c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.528175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4009390-688e-4e1a-aea0-3c740fce0c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.528462 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4009390-688e-4e1a-aea0-3c740fce0c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.528739 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4009390-688e-4e1a-aea0-3c740fce0c3f admin admin] PATH_INFO: `/v3/projects/30fa18a5bd184dedb69afd0a0a27dc78/users/7797b460a0da4e9293319b5a5bda625f/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.544350 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4009390-688e-4e1a-aea0-3c740fce0c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30fa18a5bd184dedb69afd0a0a27dc78, user_id=7797b460a0da4e9293319b5a5bda625f, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.547714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4009390-688e-4e1a-aea0-3c740fce0c3f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.560036 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1409/2823] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:12 2022] PUT /identity/v3/projects/30fa18a5bd184dedb69afd0a0a27dc78/users/7797b460a0da4e9293319b5a5bda625f/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:12.564113 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70818f-dc21-4671-bb2f-af168686ce13 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.573910 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70818f-dc21-4671-bb2f-af168686ce13 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.628444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb6ee1-17f9-44a8-b6c1-4fe8780d02b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.628983 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb6ee1-17f9-44a8-b6c1-4fe8780d02b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.629256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb6ee1-17f9-44a8-b6c1-4fe8780d02b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.629529 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb6ee1-17f9-44a8-b6c1-4fe8780d02b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.630263 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb6ee1-17f9-44a8-b6c1-4fe8780d02b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.632355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb6ee1-17f9-44a8-b6c1-4fe8780d02b3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.642004 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1415/2824] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:12.646124 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4009390-688e-4e1a-aea0-3c740fce0c3f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.656937 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4009390-688e-4e1a-aea0-3c740fce0c3f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.661883 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc16bc-5c4a-4fc6-82d8-dfb2e7908ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.662570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc16bc-5c4a-4fc6-82d8-dfb2e7908ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.662853 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc16bc-5c4a-4fc6-82d8-dfb2e7908ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.663127 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc16bc-5c4a-4fc6-82d8-dfb2e7908ed7 admin admin] PATH_INFO: `/v3/projects/30fa18a5bd184dedb69afd0a0a27dc78/users/7797b460a0da4e9293319b5a5bda625f/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.665333 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc16bc-5c4a-4fc6-82d8-dfb2e7908ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30fa18a5bd184dedb69afd0a0a27dc78, user_id=7797b460a0da4e9293319b5a5bda625f, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.667984 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc16bc-5c4a-4fc6-82d8-dfb2e7908ed7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.679003 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1410/2825] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:12 2022] PUT /identity/v3/projects/30fa18a5bd184dedb69afd0a0a27dc78/users/7797b460a0da4e9293319b5a5bda625f/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:12.688942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f05f9e-5702-468d-864c-cc17acb817fe None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.689424 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f05f9e-5702-468d-864c-cc17acb817fe None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.689538 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f05f9e-5702-468d-864c-cc17acb817fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.697312 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:12.717668 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-58f05f9e-5702-468d-864c-cc17acb817fe None None] MFA Rules not processed for user `7797b460a0da4e9293319b5a5bda625f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:12.719469 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-58f05f9e-5702-468d-864c-cc17acb817fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:12.786830 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1416/2826] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:12.798821 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc16bc-5c4a-4fc6-82d8-dfb2e7908ed7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:12.800654 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2bfc16bc-5c4a-4fc6-82d8-dfb2e7908ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:12.859332 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc16bc-5c4a-4fc6-82d8-dfb2e7908ed7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:12.863468 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2300e3-85d2-4b32-802f-d85a370a78ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:12.864082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2300e3-85d2-4b32-802f-d85a370a78ac service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:12.864363 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2300e3-85d2-4b32-802f-d85a370a78ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:12.864636 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2300e3-85d2-4b32-802f-d85a370a78ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:12.874233 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2300e3-85d2-4b32-802f-d85a370a78ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:12.876088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2300e3-85d2-4b32-802f-d85a370a78ac service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:12.885854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1411/2827] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:12.951927 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f05f9e-5702-468d-864c-cc17acb817fe None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:13.013138 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f05f9e-5702-468d-864c-cc17acb817fe None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:13.018271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:13.018936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:13.019213 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:13.019483 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] PATH_INFO: `/v3/users/fe1a95e1d3ad4b6f87657b9d5c0e7e02` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:13.020788 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe1a95e1d3ad4b6f87657b9d5c0e7e02)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:13.022890 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:13.068949 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe1a95e1d3ad4b6f87657b9d5c0e7e02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.075690 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe1a95e1d3ad4b6f87657b9d5c0e7e02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.076362 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe1a95e1d3ad4b6f87657b9d5c0e7e02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.076968 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe1a95e1d3ad4b6f87657b9d5c0e7e02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.089395 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe1a95e1d3ad4b6f87657b9d5c0e7e02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.102528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1417/2828] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:12 2022] DELETE /identity/v3/users/fe1a95e1d3ad4b6f87657b9d5c0e7e02 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:13.107114 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedb198-081d-4fe4-9fca-7898ed36bda4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:13.107398 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedb198-081d-4fe4-9fca-7898ed36bda4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:13.107676 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedb198-081d-4fe4-9fca-7898ed36bda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:13.126238 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0cedb198-081d-4fe4-9fca-7898ed36bda4 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:13.127478 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0cedb198-081d-4fe4-9fca-7898ed36bda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:13.152997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:13.188311 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1412/2829] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:13.198193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:13.199619 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:13.211349 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:13.259444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d5e3-8261-46f8-9303-801fe1f206c2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:13.263130 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43915d-2069-4ed9-b1a3-0d00fa479f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:13.263741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43915d-2069-4ed9-b1a3-0d00fa479f97 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:13.264020 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43915d-2069-4ed9-b1a3-0d00fa479f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:13.264294 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43915d-2069-4ed9-b1a3-0d00fa479f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:13.274090 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f43915d-2069-4ed9-b1a3-0d00fa479f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:13.275558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f43915d-2069-4ed9-b1a3-0d00fa479f97 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:13.285060 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1418/2830] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:13.557710 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedb198-081d-4fe4-9fca-7898ed36bda4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:13.559169 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0cedb198-081d-4fe4-9fca-7898ed36bda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:13.578308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedb198-081d-4fe4-9fca-7898ed36bda4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:13.583616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:13.584263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:13.584448 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:13.584725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] PATH_INFO: `/v3/projects/407a3aaa2c404cbd8a65ad1feb3cd682` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:13.585890 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=407a3aaa2c404cbd8a65ad1feb3cd682)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:13.587826 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:13.635229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407a3aaa2c404cbd8a65ad1feb3cd682', 'request_id': 'req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.643300 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407a3aaa2c404cbd8a65ad1feb3cd682', 'request_id': 'req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.649113 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407a3aaa2c404cbd8a65ad1feb3cd682', 'request_id': 'req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.660187 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] The token cache is being invalidate because project 407a3aaa2c404cbd8a65ad1feb3cd682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:13.660470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.661086 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.662600 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1413/2831] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:13 2022] DELETE /identity/v3/projects/407a3aaa2c404cbd8a65ad1feb3cd682 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:13.666879 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43915d-2069-4ed9-b1a3-0d00fa479f97 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:13.677739 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43915d-2069-4ed9-b1a3-0d00fa479f97 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:13.733266 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:13.733915 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:13.734159 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:13.735179 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] PATH_INFO: `/v3/users/7797b460a0da4e9293319b5a5bda625f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:13.736627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7797b460a0da4e9293319b5a5bda625f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:13.738558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:13.784314 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7797b460a0da4e9293319b5a5bda625f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.791717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7797b460a0da4e9293319b5a5bda625f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.792383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7797b460a0da4e9293319b5a5bda625f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.792988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7797b460a0da4e9293319b5a5bda625f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.805411 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-831001c9-767b-424c-bd54-fdbd4a4ff206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7797b460a0da4e9293319b5a5bda625f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:13.818387 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1419/2832] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:13 2022] DELETE /identity/v3/users/7797b460a0da4e9293319b5a5bda625f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:14.091852 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:14.093438 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:14.108084 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ce295-f108-4dd7-8d5d-c87f0f2a2dec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:14.166498 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:14.167141 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.167421 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.167697 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] PATH_INFO: `/v3/projects/30fa18a5bd184dedb69afd0a0a27dc78` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.168879 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] RBAC: Authorizing `identity:delete_project(project_id=30fa18a5bd184dedb69afd0a0a27dc78)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:14.170830 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:14.220312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30fa18a5bd184dedb69afd0a0a27dc78', 'request_id': 'req-6c54d632-369d-4b51-8401-347ba803410c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:14.227563 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30fa18a5bd184dedb69afd0a0a27dc78', 'request_id': 'req-6c54d632-369d-4b51-8401-347ba803410c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:14.228771 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:14.234202 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30fa18a5bd184dedb69afd0a0a27dc78', 'request_id': 'req-6c54d632-369d-4b51-8401-347ba803410c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:14.255544 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] The token cache is being invalidate because project 30fa18a5bd184dedb69afd0a0a27dc78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:14.255836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:14.256482 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c54d632-369d-4b51-8401-347ba803410c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:14.257993 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1414/2833] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:14 2022] DELETE /identity/v3/projects/30fa18a5bd184dedb69afd0a0a27dc78 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:14.263768 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc808cc-2822-4467-916b-57bf24289c35 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.264052 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc808cc-2822-4467-916b-57bf24289c35 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.264320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc808cc-2822-4467-916b-57bf24289c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.284114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bdc808cc-2822-4467-916b-57bf24289c35 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:14.285320 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bdc808cc-2822-4467-916b-57bf24289c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:14.311578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:14.348631 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1420/2834] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:14.412570 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af979f-2aeb-44d0-ae39-ca310d802337 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.412856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af979f-2aeb-44d0-ae39-ca310d802337 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.413129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af979f-2aeb-44d0-ae39-ca310d802337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.433645 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d8af979f-2aeb-44d0-ae39-ca310d802337 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:14.434884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d8af979f-2aeb-44d0-ae39-ca310d802337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:14.443427 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1415/2835] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:14.447640 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc808cc-2822-4467-916b-57bf24289c35 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:14.457233 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc808cc-2822-4467-916b-57bf24289c35 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:14.461760 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f47d-8ed4-44c3-9e32-6158ada7d14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:14.462357 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f47d-8ed4-44c3-9e32-6158ada7d14c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.462697 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f47d-8ed4-44c3-9e32-6158ada7d14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.462992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278f47d-8ed4-44c3-9e32-6158ada7d14c admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.463670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f47d-8ed4-44c3-9e32-6158ada7d14c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:10:14.463966 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f47d-8ed4-44c3-9e32-6158ada7d14c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:14.465571 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278f47d-8ed4-44c3-9e32-6158ada7d14c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:14.476881 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1421/2836] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:10:14.480896 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af979f-2aeb-44d0-ae39-ca310d802337 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:14.490578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af979f-2aeb-44d0-ae39-ca310d802337 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:14.495175 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c82374-2c78-451b-87cd-71e94549ef84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:14.495764 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c82374-2c78-451b-87cd-71e94549ef84 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.496039 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c82374-2c78-451b-87cd-71e94549ef84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.496308 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c82374-2c78-451b-87cd-71e94549ef84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.496952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c82374-2c78-451b-87cd-71e94549ef84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:14.498878 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c82374-2c78-451b-87cd-71e94549ef84 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:14.526419 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1416/2837] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:14 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:14.530622 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f47d-8ed4-44c3-9e32-6158ada7d14c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:14.540689 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278f47d-8ed4-44c3-9e32-6158ada7d14c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:14.595524 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b1d2b-1fa7-4b0c-979f-39f4262c2042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:14.596128 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b1d2b-1fa7-4b0c-979f-39f4262c2042 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.596406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b1d2b-1fa7-4b0c-979f-39f4262c2042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.596678 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b1d2b-1fa7-4b0c-979f-39f4262c2042 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.597348 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b1d2b-1fa7-4b0c-979f-39f4262c2042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:14.599265 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b1d2b-1fa7-4b0c-979f-39f4262c2042 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:14.620273 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1422/2838] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:14 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:14.624461 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c82374-2c78-451b-87cd-71e94549ef84 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:14.635517 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c82374-2c78-451b-87cd-71e94549ef84 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:14.640241 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43bb03-87cd-4528-b4f5-c8bdd5b74cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:14.640956 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43bb03-87cd-4528-b4f5-c8bdd5b74cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.641239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43bb03-87cd-4528-b4f5-c8bdd5b74cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.641516 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43bb03-87cd-4528-b4f5-c8bdd5b74cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.642206 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43bb03-87cd-4528-b4f5-c8bdd5b74cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:14.643857 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43bb03-87cd-4528-b4f5-c8bdd5b74cf1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:14.653666 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1417/2839] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:14.658092 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b1d2b-1fa7-4b0c-979f-39f4262c2042 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:14.668396 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b1d2b-1fa7-4b0c-979f-39f4262c2042 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:14.673557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a883d-3c11-4600-aed7-02c2fb282ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:14.674149 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a883d-3c11-4600-aed7-02c2fb282ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.674428 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a883d-3c11-4600-aed7-02c2fb282ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.674727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a883d-3c11-4600-aed7-02c2fb282ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.675444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a883d-3c11-4600-aed7-02c2fb282ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:14.677007 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a883d-3c11-4600-aed7-02c2fb282ac8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:14.687554 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1423/2840] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 30 05:10:14.691850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43bb03-87cd-4528-b4f5-c8bdd5b74cf1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:14.701827 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43bb03-87cd-4528-b4f5-c8bdd5b74cf1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:14.706819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8df9-c9d7-4f26-8abf-523bcc9f0b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:14.707525 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8df9-c9d7-4f26-8abf-523bcc9f0b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.707813 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8df9-c9d7-4f26-8abf-523bcc9f0b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.708091 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8df9-c9d7-4f26-8abf-523bcc9f0b6d admin admin] PATH_INFO: `/v3/projects/b1d5f94c57964eb3aaf513abcaa2859a/users/f32a5e89dbc74823a67a111e6337461d/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.723189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea8df9-c9d7-4f26-8abf-523bcc9f0b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d5f94c57964eb3aaf513abcaa2859a, user_id=f32a5e89dbc74823a67a111e6337461d, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:14.725854 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea8df9-c9d7-4f26-8abf-523bcc9f0b6d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:14.739560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1418/2841] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:14 2022] PUT /identity/v3/projects/b1d5f94c57964eb3aaf513abcaa2859a/users/f32a5e89dbc74823a67a111e6337461d/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:10:14.749439 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121470b-6a9b-47bc-9052-fd35beccd301 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.749953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121470b-6a9b-47bc-9052-fd35beccd301 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.750135 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121470b-6a9b-47bc-9052-fd35beccd301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.777471 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2121470b-6a9b-47bc-9052-fd35beccd301 None None] MFA Rules not processed for user `f32a5e89dbc74823a67a111e6337461d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:14.778734 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2121470b-6a9b-47bc-9052-fd35beccd301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:14.841644 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1424/2842] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:14.846831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8df9-c9d7-4f26-8abf-523bcc9f0b6d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:14.909346 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8df9-c9d7-4f26-8abf-523bcc9f0b6d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:14.914220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bec8d1-7a7f-491e-b8db-009896119316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:14.914860 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bec8d1-7a7f-491e-b8db-009896119316 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:14.915136 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bec8d1-7a7f-491e-b8db-009896119316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:14.915406 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bec8d1-7a7f-491e-b8db-009896119316 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:14.916058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bec8d1-7a7f-491e-b8db-009896119316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:14.917949 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bec8d1-7a7f-491e-b8db-009896119316 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:14.944118 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1419/2843] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:14 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:14.948148 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121470b-6a9b-47bc-9052-fd35beccd301 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:14.959303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121470b-6a9b-47bc-9052-fd35beccd301 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:14.982869 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:15.017057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d2212-e7ed-4f88-a1a1-96e2d44478dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:15.017617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d2212-e7ed-4f88-a1a1-96e2d44478dd admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.017894 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d2212-e7ed-4f88-a1a1-96e2d44478dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.018161 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d2212-e7ed-4f88-a1a1-96e2d44478dd admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.018862 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d2212-e7ed-4f88-a1a1-96e2d44478dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:15.020908 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d2212-e7ed-4f88-a1a1-96e2d44478dd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:15.041463 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1425/2844] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:14 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:15.045424 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bec8d1-7a7f-491e-b8db-009896119316 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:15.056307 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bec8d1-7a7f-491e-b8db-009896119316 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:15.060850 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509ff87-6e47-4eb9-93d9-b71e18c367fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:15.061449 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509ff87-6e47-4eb9-93d9-b71e18c367fe admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.061726 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509ff87-6e47-4eb9-93d9-b71e18c367fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.061997 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509ff87-6e47-4eb9-93d9-b71e18c367fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.063316 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509ff87-6e47-4eb9-93d9-b71e18c367fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:15.065000 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509ff87-6e47-4eb9-93d9-b71e18c367fe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:15.075387 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1420/2845] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:15.079366 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d2212-e7ed-4f88-a1a1-96e2d44478dd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:15.089440 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d2212-e7ed-4f88-a1a1-96e2d44478dd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:15.094095 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7136-3f25-4bc6-b1b7-da0bba1b74e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:15.094756 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7136-3f25-4bc6-b1b7-da0bba1b74e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.095032 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7136-3f25-4bc6-b1b7-da0bba1b74e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.095301 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7136-3f25-4bc6-b1b7-da0bba1b74e6 admin admin] PATH_INFO: `/v3/projects/4a8b5690f730494d8f005f7be71536bb/users/64765d1ff39d4dbeb2e7516a56372ed6/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.110073 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad7136-3f25-4bc6-b1b7-da0bba1b74e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8b5690f730494d8f005f7be71536bb, user_id=64765d1ff39d4dbeb2e7516a56372ed6, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:15.112727 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad7136-3f25-4bc6-b1b7-da0bba1b74e6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:15.126214 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1426/2846] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:15 2022] PUT /identity/v3/projects/4a8b5690f730494d8f005f7be71536bb/users/64765d1ff39d4dbeb2e7516a56372ed6/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:15.130701 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509ff87-6e47-4eb9-93d9-b71e18c367fe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:15.140690 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509ff87-6e47-4eb9-93d9-b71e18c367fe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:15.190271 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:15.198493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6935d07-b605-4f14-8e5e-a1319ef4ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:15.199278 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6935d07-b605-4f14-8e5e-a1319ef4ea42 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.199718 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6935d07-b605-4f14-8e5e-a1319ef4ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.200043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6935d07-b605-4f14-8e5e-a1319ef4ea42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.200894 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6935d07-b605-4f14-8e5e-a1319ef4ea42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:15.202578 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6935d07-b605-4f14-8e5e-a1319ef4ea42 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:15.212745 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1421/2847] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:15.216911 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7136-3f25-4bc6-b1b7-da0bba1b74e6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:15.227350 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7136-3f25-4bc6-b1b7-da0bba1b74e6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:15.231929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0f56a-243c-4521-8140-17ce3cdc821f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:15.232578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0f56a-243c-4521-8140-17ce3cdc821f admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.232855 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0f56a-243c-4521-8140-17ce3cdc821f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.233129 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0f56a-243c-4521-8140-17ce3cdc821f admin admin] PATH_INFO: `/v3/projects/4a8b5690f730494d8f005f7be71536bb/users/64765d1ff39d4dbeb2e7516a56372ed6/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.235674 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0f56a-243c-4521-8140-17ce3cdc821f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8b5690f730494d8f005f7be71536bb, user_id=64765d1ff39d4dbeb2e7516a56372ed6, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:15.238303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0f56a-243c-4521-8140-17ce3cdc821f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:15.248264 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1427/2848] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:15 2022] PUT /identity/v3/projects/4a8b5690f730494d8f005f7be71536bb/users/64765d1ff39d4dbeb2e7516a56372ed6/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:15.260002 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c14390-9f06-4467-8aa9-6c82cc5a3aea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.260289 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c14390-9f06-4467-8aa9-6c82cc5a3aea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.260567 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c14390-9f06-4467-8aa9-6c82cc5a3aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.289040 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-57c14390-9f06-4467-8aa9-6c82cc5a3aea None None] MFA Rules not processed for user `64765d1ff39d4dbeb2e7516a56372ed6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:15.290201 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-57c14390-9f06-4467-8aa9-6c82cc5a3aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:15.357882 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1422/2849] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:15.369177 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0f56a-243c-4521-8140-17ce3cdc821f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:15.370980 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4ae0f56a-243c-4521-8140-17ce3cdc821f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:15.429256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0f56a-243c-4521-8140-17ce3cdc821f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:15.432953 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d30ce-5bc5-4b76-89c8-1bb87a231233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:15.433557 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d30ce-5bc5-4b76-89c8-1bb87a231233 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.433834 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d30ce-5bc5-4b76-89c8-1bb87a231233 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.434106 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d30ce-5bc5-4b76-89c8-1bb87a231233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.443423 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d30ce-5bc5-4b76-89c8-1bb87a231233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:15.444851 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d30ce-5bc5-4b76-89c8-1bb87a231233 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:15.454467 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1428/2850] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:15.592616 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c14390-9f06-4467-8aa9-6c82cc5a3aea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:15.655906 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c14390-9f06-4467-8aa9-6c82cc5a3aea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:15.661134 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:15.661765 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.662042 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.662312 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] PATH_INFO: `/v3/users/f32a5e89dbc74823a67a111e6337461d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.663505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32a5e89dbc74823a67a111e6337461d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:15.665672 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:15.713216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32a5e89dbc74823a67a111e6337461d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:15.720959 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32a5e89dbc74823a67a111e6337461d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:15.721621 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32a5e89dbc74823a67a111e6337461d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:15.722239 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32a5e89dbc74823a67a111e6337461d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:15.735253 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32a5e89dbc74823a67a111e6337461d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:15.739087 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:15.750981 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1423/2851] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:15 2022] DELETE /identity/v3/users/f32a5e89dbc74823a67a111e6337461d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:15.755517 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0d9ac-7054-4eed-bcd8-da211a79136c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.755988 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0d9ac-7054-4eed-bcd8-da211a79136c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.756085 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac0d9ac-7054-4eed-bcd8-da211a79136c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.774925 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0ac0d9ac-7054-4eed-bcd8-da211a79136c None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:15.776111 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0ac0d9ac-7054-4eed-bcd8-da211a79136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:15.834398 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1429/2852] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:15.844822 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:15.846220 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:15.903432 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daa514-b77a-478d-a363-595dcb46b80d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:15.907668 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9eef2-cf7e-4c6a-9850-6068a6c3b119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:15.908272 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9eef2-cf7e-4c6a-9850-6068a6c3b119 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:15.908550 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9eef2-cf7e-4c6a-9850-6068a6c3b119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:15.908819 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9eef2-cf7e-4c6a-9850-6068a6c3b119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:15.917910 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9eef2-cf7e-4c6a-9850-6068a6c3b119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:15.919351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9eef2-cf7e-4c6a-9850-6068a6c3b119 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:15.928702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1424/2853] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:16.202629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0d9ac-7054-4eed-bcd8-da211a79136c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:16.204904 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0ac0d9ac-7054-4eed-bcd8-da211a79136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:16.216817 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac0d9ac-7054-4eed-bcd8-da211a79136c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:16.222169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:16.222824 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:16.223086 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:16.223609 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] PATH_INFO: `/v3/projects/b1d5f94c57964eb3aaf513abcaa2859a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:16.224482 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d5f94c57964eb3aaf513abcaa2859a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:16.226409 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:16.244578 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:16.276231 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d5f94c57964eb3aaf513abcaa2859a', 'request_id': 'req-b6e545a5-39b6-4538-b302-807d938498f2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.283777 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d5f94c57964eb3aaf513abcaa2859a', 'request_id': 'req-b6e545a5-39b6-4538-b302-807d938498f2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.290181 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d5f94c57964eb3aaf513abcaa2859a', 'request_id': 'req-b6e545a5-39b6-4538-b302-807d938498f2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.301343 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] The token cache is being invalidate because project b1d5f94c57964eb3aaf513abcaa2859a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:16.301624 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.302274 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.303813 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1430/2854] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:16 2022] DELETE /identity/v3/projects/b1d5f94c57964eb3aaf513abcaa2859a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:16.307903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9eef2-cf7e-4c6a-9850-6068a6c3b119 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:16.318661 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9eef2-cf7e-4c6a-9850-6068a6c3b119 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:16.376493 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:16.377076 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:16.377357 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:16.377630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] PATH_INFO: `/v3/users/64765d1ff39d4dbeb2e7516a56372ed6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:16.378831 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] RBAC: Authorizing `identity:delete_user(user_id=64765d1ff39d4dbeb2e7516a56372ed6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:16.380748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:16.426054 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64765d1ff39d4dbeb2e7516a56372ed6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.433192 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64765d1ff39d4dbeb2e7516a56372ed6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.433845 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64765d1ff39d4dbeb2e7516a56372ed6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.434481 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64765d1ff39d4dbeb2e7516a56372ed6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.446748 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebef6989-f58c-42d4-b4dd-1948f576cfda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64765d1ff39d4dbeb2e7516a56372ed6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.459695 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1425/2855] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:16 2022] DELETE /identity/v3/users/64765d1ff39d4dbeb2e7516a56372ed6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:16.737239 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:16.738835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:16.756564 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e545a5-39b6-4538-b302-807d938498f2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:16.814300 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:16.814990 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:16.815262 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:16.815528 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] PATH_INFO: `/v3/projects/4a8b5690f730494d8f005f7be71536bb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:16.816658 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8b5690f730494d8f005f7be71536bb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:16.818575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:16.864259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8b5690f730494d8f005f7be71536bb', 'request_id': 'req-6f783ecb-989b-4a3c-8be0-667da0ec973b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.871331 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8b5690f730494d8f005f7be71536bb', 'request_id': 'req-6f783ecb-989b-4a3c-8be0-667da0ec973b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.877652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8b5690f730494d8f005f7be71536bb', 'request_id': 'req-6f783ecb-989b-4a3c-8be0-667da0ec973b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.888420 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] The token cache is being invalidate because project 4a8b5690f730494d8f005f7be71536bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:16.888705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.889339 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6f783ecb-989b-4a3c-8be0-667da0ec973b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:16.891638 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1431/2856] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:16 2022] DELETE /identity/v3/projects/4a8b5690f730494d8f005f7be71536bb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:16.896630 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef7b79-f657-490f-8564-9db7db745f4e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:16.896947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef7b79-f657-490f-8564-9db7db745f4e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:16.897219 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef7b79-f657-490f-8564-9db7db745f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:16.917907 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d6ef7b79-f657-490f-8564-9db7db745f4e None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:16.919083 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d6ef7b79-f657-490f-8564-9db7db745f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:16.931275 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:16.981952 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1426/2857] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:16.989791 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4e58-44d6-4777-b068-b711e8a95ab3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:16.990075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4e58-44d6-4777-b068-b711e8a95ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:16.990351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4e58-44d6-4777-b068-b711e8a95ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:17.010514 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-272d4e58-44d6-4777-b068-b711e8a95ab3 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:17.011705 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-272d4e58-44d6-4777-b068-b711e8a95ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:17.020490 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1432/2858] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:17.024783 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef7b79-f657-490f-8564-9db7db745f4e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:17.034974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef7b79-f657-490f-8564-9db7db745f4e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:17.039866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad599a5-89a7-49a8-beeb-599eb82f6d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:17.040472 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad599a5-89a7-49a8-beeb-599eb82f6d90 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:17.040751 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad599a5-89a7-49a8-beeb-599eb82f6d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:17.041019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad599a5-89a7-49a8-beeb-599eb82f6d90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:17.041731 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad599a5-89a7-49a8-beeb-599eb82f6d90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:10:17.042029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad599a5-89a7-49a8-beeb-599eb82f6d90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:17.043641 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad599a5-89a7-49a8-beeb-599eb82f6d90 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:17.055400 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1427/2859] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:10:17.059580 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4e58-44d6-4777-b068-b711e8a95ab3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:17.069384 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4e58-44d6-4777-b068-b711e8a95ab3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:17.074021 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7a75f-f705-4b2e-84ea-dc6e1bca1af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:17.074652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7a75f-f705-4b2e-84ea-dc6e1bca1af3 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:17.074929 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7a75f-f705-4b2e-84ea-dc6e1bca1af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:17.075198 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7a75f-f705-4b2e-84ea-dc6e1bca1af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:17.075866 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7a75f-f705-4b2e-84ea-dc6e1bca1af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:17.077757 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7a75f-f705-4b2e-84ea-dc6e1bca1af3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:17.104967 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1433/2860] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:17 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:17.109055 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad599a5-89a7-49a8-beeb-599eb82f6d90 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:17.119044 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad599a5-89a7-49a8-beeb-599eb82f6d90 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:17.175507 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcbc4f-b7d2-403d-98b2-d75b90d32fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:17.176041 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcbc4f-b7d2-403d-98b2-d75b90d32fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:17.176318 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcbc4f-b7d2-403d-98b2-d75b90d32fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:17.176587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcbc4f-b7d2-403d-98b2-d75b90d32fe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:17.177264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcbc4f-b7d2-403d-98b2-d75b90d32fe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:17.179196 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcbc4f-b7d2-403d-98b2-d75b90d32fe3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:17.201163 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1428/2861] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:17 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:10:17.205825 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7a75f-f705-4b2e-84ea-dc6e1bca1af3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:17.216600 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7a75f-f705-4b2e-84ea-dc6e1bca1af3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:17.221402 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1e9d3-5406-44e6-8941-a27634366067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:17.221998 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1e9d3-5406-44e6-8941-a27634366067 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:17.222271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1e9d3-5406-44e6-8941-a27634366067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:17.222575 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1e9d3-5406-44e6-8941-a27634366067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:17.223259 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1e9d3-5406-44e6-8941-a27634366067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:17.224829 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1e9d3-5406-44e6-8941-a27634366067 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:17.226051 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:17.236793 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1434/2862] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:17.240918 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcbc4f-b7d2-403d-98b2-d75b90d32fe3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:17.251587 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcbc4f-b7d2-403d-98b2-d75b90d32fe3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:17.256263 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2ea80-2292-4a18-af14-b1203043cfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:17.256870 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2ea80-2292-4a18-af14-b1203043cfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:17.257154 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2ea80-2292-4a18-af14-b1203043cfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:17.257432 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2ea80-2292-4a18-af14-b1203043cfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:17.258132 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2ea80-2292-4a18-af14-b1203043cfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:17.259781 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2ea80-2292-4a18-af14-b1203043cfb1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:17.270756 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1429/2863] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:17.274696 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1e9d3-5406-44e6-8941-a27634366067 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:17.284656 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1e9d3-5406-44e6-8941-a27634366067 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:17.289462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577c2e9-2c97-4534-b22e-d0bc5539eba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:17.290104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577c2e9-2c97-4534-b22e-d0bc5539eba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:17.290378 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577c2e9-2c97-4534-b22e-d0bc5539eba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:17.290709 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577c2e9-2c97-4534-b22e-d0bc5539eba7 admin admin] PATH_INFO: `/v3/projects/9fd9cb4b605f4c0e8f18e630332594f8/users/2438dc0daa874fe88edfbf7b4792befb/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:17.306470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577c2e9-2c97-4534-b22e-d0bc5539eba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd9cb4b605f4c0e8f18e630332594f8, user_id=2438dc0daa874fe88edfbf7b4792befb, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:17.309075 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577c2e9-2c97-4534-b22e-d0bc5539eba7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:17.322414 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1435/2864] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:17 2022] PUT /identity/v3/projects/9fd9cb4b605f4c0e8f18e630332594f8/users/2438dc0daa874fe88edfbf7b4792befb/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:17.332974 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6efcc-c2f4-4bd5-8fc8-4d2c38a4dfd3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:17.333465 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6efcc-c2f4-4bd5-8fc8-4d2c38a4dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:17.333655 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6efcc-c2f4-4bd5-8fc8-4d2c38a4dfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:17.361593 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dce6efcc-c2f4-4bd5-8fc8-4d2c38a4dfd3 None None] MFA Rules not processed for user `2438dc0daa874fe88edfbf7b4792befb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:17.362856 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dce6efcc-c2f4-4bd5-8fc8-4d2c38a4dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:17.426979 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1430/2865] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:17.438573 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577c2e9-2c97-4534-b22e-d0bc5539eba7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:17.440271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4577c2e9-2c97-4534-b22e-d0bc5539eba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:17.496368 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577c2e9-2c97-4534-b22e-d0bc5539eba7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:17.501046 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7cd58-85c8-40e2-89fa-d9f0cae03eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:17.501667 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7cd58-85c8-40e2-89fa-d9f0cae03eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:17.501959 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7cd58-85c8-40e2-89fa-d9f0cae03eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:17.502219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7cd58-85c8-40e2-89fa-d9f0cae03eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:17.510706 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7cd58-85c8-40e2-89fa-d9f0cae03eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:17.512141 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7cd58-85c8-40e2-89fa-d9f0cae03eb9 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:17.521103 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1436/2866] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:27.424082 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:27.517176 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:35.086682 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6efcc-c2f4-4bd5-8fc8-4d2c38a4dfd3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:35.148656 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6efcc-c2f4-4bd5-8fc8-4d2c38a4dfd3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:35.153454 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:35.154088 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:35.154365 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:35.154663 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] PATH_INFO: `/v3/users/2438dc0daa874fe88edfbf7b4792befb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:35.155820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2438dc0daa874fe88edfbf7b4792befb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:35.158714 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:35.205255 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2438dc0daa874fe88edfbf7b4792befb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.212256 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2438dc0daa874fe88edfbf7b4792befb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.212943 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2438dc0daa874fe88edfbf7b4792befb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.213579 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2438dc0daa874fe88edfbf7b4792befb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.226674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2438dc0daa874fe88edfbf7b4792befb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.240063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1431/2867] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:35 2022] DELETE /identity/v3/users/2438dc0daa874fe88edfbf7b4792befb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:35.244590 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308b1c0-2fa1-45ff-a1be-270595ca29b0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:35.245101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308b1c0-2fa1-45ff-a1be-270595ca29b0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:35.245222 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308b1c0-2fa1-45ff-a1be-270595ca29b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:35.264303 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e308b1c0-2fa1-45ff-a1be-270595ca29b0 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:35.265494 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e308b1c0-2fa1-45ff-a1be-270595ca29b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:35.328219 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1437/2868] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:35.339129 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:35.340560 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:35.398601 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5af3de-de9e-49ed-ab78-77ed75d7a86c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:35.402268 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd814c5-e790-465d-b357-6c54ae1d8ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:35.402904 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd814c5-e790-465d-b357-6c54ae1d8ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:35.403189 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd814c5-e790-465d-b357-6c54ae1d8ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:35.403466 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd814c5-e790-465d-b357-6c54ae1d8ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:35.412629 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd814c5-e790-465d-b357-6c54ae1d8ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:35.414319 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd814c5-e790-465d-b357-6c54ae1d8ec4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:35.423884 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1432/2869] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:35.684002 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308b1c0-2fa1-45ff-a1be-270595ca29b0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:35.685570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e308b1c0-2fa1-45ff-a1be-270595ca29b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:35.698694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308b1c0-2fa1-45ff-a1be-270595ca29b0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:35.703340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:35.704104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:35.704351 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:35.704782 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] PATH_INFO: `/v3/projects/9fd9cb4b605f4c0e8f18e630332594f8` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:35.705973 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd9cb4b605f4c0e8f18e630332594f8)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:35.707917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:35.754874 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd9cb4b605f4c0e8f18e630332594f8', 'request_id': 'req-f20d056d-698e-455b-93e9-4be3828d8f7e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.763076 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd9cb4b605f4c0e8f18e630332594f8', 'request_id': 'req-f20d056d-698e-455b-93e9-4be3828d8f7e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.768516 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd9cb4b605f4c0e8f18e630332594f8', 'request_id': 'req-f20d056d-698e-455b-93e9-4be3828d8f7e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.779627 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] The token cache is being invalidate because project 9fd9cb4b605f4c0e8f18e630332594f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:35.779917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.780545 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f20d056d-698e-455b-93e9-4be3828d8f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:35.782126 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1438/2870] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:35 2022] DELETE /identity/v3/projects/9fd9cb4b605f4c0e8f18e630332594f8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:35.789211 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a06b15-4ef6-4049-b8ea-4837757a05e3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:35.789505 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a06b15-4ef6-4049-b8ea-4837757a05e3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:35.789784 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a06b15-4ef6-4049-b8ea-4837757a05e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:35.809589 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-65a06b15-4ef6-4049-b8ea-4837757a05e3 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:35.810842 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65a06b15-4ef6-4049-b8ea-4837757a05e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:35.828225 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:35.873354 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1433/2871] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:35.881935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffaf23-7b62-497e-b832-d6a6fa32500f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:35.882218 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffaf23-7b62-497e-b832-d6a6fa32500f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:35.882523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffaf23-7b62-497e-b832-d6a6fa32500f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:35.901717 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dcffaf23-7b62-497e-b832-d6a6fa32500f None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:35.902940 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dcffaf23-7b62-497e-b832-d6a6fa32500f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:35.910958 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1439/2872] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:35.915436 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a06b15-4ef6-4049-b8ea-4837757a05e3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:35.925627 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a06b15-4ef6-4049-b8ea-4837757a05e3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:35.930174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179e6e7-ea2c-4fe7-bb51-9de0f6ec0939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:35.930818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179e6e7-ea2c-4fe7-bb51-9de0f6ec0939 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:35.931097 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179e6e7-ea2c-4fe7-bb51-9de0f6ec0939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:35.931370 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179e6e7-ea2c-4fe7-bb51-9de0f6ec0939 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:35.932061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179e6e7-ea2c-4fe7-bb51-9de0f6ec0939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 30 05:10:35.932360 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179e6e7-ea2c-4fe7-bb51-9de0f6ec0939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:35.933958 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179e6e7-ea2c-4fe7-bb51-9de0f6ec0939 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:35.945961 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1434/2873] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Jul 30 05:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 30 05:10:35.949992 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffaf23-7b62-497e-b832-d6a6fa32500f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:35.959313 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffaf23-7b62-497e-b832-d6a6fa32500f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:35.963917 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64da27a-c74f-4cf0-bd2b-f4cbb5064aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:35.964508 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64da27a-c74f-4cf0-bd2b-f4cbb5064aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:35.964781 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64da27a-c74f-4cf0-bd2b-f4cbb5064aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:35.965049 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64da27a-c74f-4cf0-bd2b-f4cbb5064aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:35.965737 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64da27a-c74f-4cf0-bd2b-f4cbb5064aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:35.967642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64da27a-c74f-4cf0-bd2b-f4cbb5064aaf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:35.994406 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1440/2874] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:35 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:35.998561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179e6e7-ea2c-4fe7-bb51-9de0f6ec0939 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.009511 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179e6e7-ea2c-4fe7-bb51-9de0f6ec0939 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.068058 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212eba6-54c7-41d1-9c3c-d18aeb7dc7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.068689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212eba6-54c7-41d1-9c3c-d18aeb7dc7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.068973 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212eba6-54c7-41d1-9c3c-d18aeb7dc7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.069254 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212eba6-54c7-41d1-9c3c-d18aeb7dc7a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.069929 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212eba6-54c7-41d1-9c3c-d18aeb7dc7a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.071886 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212eba6-54c7-41d1-9c3c-d18aeb7dc7a1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.092216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1435/2875] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:35 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:36.097025 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64da27a-c74f-4cf0-bd2b-f4cbb5064aaf admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.107194 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64da27a-c74f-4cf0-bd2b-f4cbb5064aaf admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.112550 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1aeef-f95f-4e3c-a3a2-1202110610bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.113145 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1aeef-f95f-4e3c-a3a2-1202110610bb admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.113421 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1aeef-f95f-4e3c-a3a2-1202110610bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.113687 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1aeef-f95f-4e3c-a3a2-1202110610bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.114371 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1aeef-f95f-4e3c-a3a2-1202110610bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.115975 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1aeef-f95f-4e3c-a3a2-1202110610bb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.127226 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1441/2876] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:36.131148 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212eba6-54c7-41d1-9c3c-d18aeb7dc7a1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.141434 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212eba6-54c7-41d1-9c3c-d18aeb7dc7a1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.146063 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f60158-b1c2-487d-8b31-3574746b05b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.146699 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f60158-b1c2-487d-8b31-3574746b05b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.147032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f60158-b1c2-487d-8b31-3574746b05b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.147245 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f60158-b1c2-487d-8b31-3574746b05b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.148064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f60158-b1c2-487d-8b31-3574746b05b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.149778 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f60158-b1c2-487d-8b31-3574746b05b5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.160903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1436/2877] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:36.165193 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1aeef-f95f-4e3c-a3a2-1202110610bb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.175652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1aeef-f95f-4e3c-a3a2-1202110610bb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.180196 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590d3aa-1824-4792-847a-c7e042566b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.180840 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590d3aa-1824-4792-847a-c7e042566b96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.181115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590d3aa-1824-4792-847a-c7e042566b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.181383 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590d3aa-1824-4792-847a-c7e042566b96 admin admin] PATH_INFO: `/v3/projects/8ec7e37cf7e845c79534c5f8955af5ee/users/3fee48aadc65451a903ef132bc5c14ff/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.195645 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590d3aa-1824-4792-847a-c7e042566b96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec7e37cf7e845c79534c5f8955af5ee, user_id=3fee48aadc65451a903ef132bc5c14ff, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.198269 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590d3aa-1824-4792-847a-c7e042566b96 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.211104 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1442/2878] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:36 2022] PUT /identity/v3/projects/8ec7e37cf7e845c79534c5f8955af5ee/users/3fee48aadc65451a903ef132bc5c14ff/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:36.221623 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2548c-6b3a-4717-a386-67c9259fdd47 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.221903 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2548c-6b3a-4717-a386-67c9259fdd47 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.222174 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2548c-6b3a-4717-a386-67c9259fdd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.249921 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-42e2548c-6b3a-4717-a386-67c9259fdd47 None None] MFA Rules not processed for user `3fee48aadc65451a903ef132bc5c14ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:36.251347 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42e2548c-6b3a-4717-a386-67c9259fdd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:36.315052 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1437/2879] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:36.319597 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590d3aa-1824-4792-847a-c7e042566b96 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.340895 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:36.385759 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590d3aa-1824-4792-847a-c7e042566b96 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.390476 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473224b5-3729-4de0-bedf-fab1b698109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.391651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473224b5-3729-4de0-bedf-fab1b698109c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.391936 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473224b5-3729-4de0-bedf-fab1b698109c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.392209 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473224b5-3729-4de0-bedf-fab1b698109c admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.392882 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473224b5-3729-4de0-bedf-fab1b698109c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.394805 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473224b5-3729-4de0-bedf-fab1b698109c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.421207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1443/2880] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:36 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:36.425412 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2548c-6b3a-4717-a386-67c9259fdd47 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.436067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2548c-6b3a-4717-a386-67c9259fdd47 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.449008 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:36.494740 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed1ddc-a467-4c6a-abd7-236df7b69858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.495351 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed1ddc-a467-4c6a-abd7-236df7b69858 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.495634 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed1ddc-a467-4c6a-abd7-236df7b69858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.495911 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed1ddc-a467-4c6a-abd7-236df7b69858 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.496585 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed1ddc-a467-4c6a-abd7-236df7b69858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.498561 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed1ddc-a467-4c6a-abd7-236df7b69858 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.520229 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1438/2881] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:36 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:36.524234 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473224b5-3729-4de0-bedf-fab1b698109c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.535067 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473224b5-3729-4de0-bedf-fab1b698109c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.540629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39286f-ee02-4b65-895a-5e2d3365b168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.541256 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39286f-ee02-4b65-895a-5e2d3365b168 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.541534 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39286f-ee02-4b65-895a-5e2d3365b168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.541804 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39286f-ee02-4b65-895a-5e2d3365b168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.542518 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39286f-ee02-4b65-895a-5e2d3365b168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.544102 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39286f-ee02-4b65-895a-5e2d3365b168 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.555244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1444/2882] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:36.559079 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed1ddc-a467-4c6a-abd7-236df7b69858 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.568965 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed1ddc-a467-4c6a-abd7-236df7b69858 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.573534 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fdafa-3509-41da-a346-84ae2eb5a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.574165 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fdafa-3509-41da-a346-84ae2eb5a85b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.574456 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fdafa-3509-41da-a346-84ae2eb5a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.574753 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fdafa-3509-41da-a346-84ae2eb5a85b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.575444 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fdafa-3509-41da-a346-84ae2eb5a85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.577028 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fdafa-3509-41da-a346-84ae2eb5a85b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.587580 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1439/2883] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:36.591462 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39286f-ee02-4b65-895a-5e2d3365b168 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.602029 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39286f-ee02-4b65-895a-5e2d3365b168 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.606614 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae83f6e-b8da-4e84-869e-ecd0b1add3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.607248 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae83f6e-b8da-4e84-869e-ecd0b1add3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.607523 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae83f6e-b8da-4e84-869e-ecd0b1add3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.607792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae83f6e-b8da-4e84-869e-ecd0b1add3a7 admin admin] PATH_INFO: `/v3/projects/88ef764f43644c279abd7b1b8f4a0a47/users/e99a817656544ccb8a696bf7340b210b/roles/3d81447d2d66443384ce9ae7dd5ea35d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.621792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae83f6e-b8da-4e84-869e-ecd0b1add3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ef764f43644c279abd7b1b8f4a0a47, user_id=e99a817656544ccb8a696bf7340b210b, role_id=3d81447d2d66443384ce9ae7dd5ea35d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.624417 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae83f6e-b8da-4e84-869e-ecd0b1add3a7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.636942 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1445/2884] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:36 2022] PUT /identity/v3/projects/88ef764f43644c279abd7b1b8f4a0a47/users/e99a817656544ccb8a696bf7340b210b/roles/3d81447d2d66443384ce9ae7dd5ea35d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 30 05:10:36.649528 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7795ac-2a04-4905-98f8-3239fcb6cf66 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.649823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7795ac-2a04-4905-98f8-3239fcb6cf66 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.650131 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7795ac-2a04-4905-98f8-3239fcb6cf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.678820 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4f7795ac-2a04-4905-98f8-3239fcb6cf66 None None] MFA Rules not processed for user `e99a817656544ccb8a696bf7340b210b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:36.680032 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f7795ac-2a04-4905-98f8-3239fcb6cf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:36.744888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1440/2885] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:36.749560 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae83f6e-b8da-4e84-869e-ecd0b1add3a7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.810340 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae83f6e-b8da-4e84-869e-ecd0b1add3a7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.815728 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c5bf2-ef58-4f55-bf46-b18c27449109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.816335 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c5bf2-ef58-4f55-bf46-b18c27449109 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.816617 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c5bf2-ef58-4f55-bf46-b18c27449109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.816892 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c5bf2-ef58-4f55-bf46-b18c27449109 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.817546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c5bf2-ef58-4f55-bf46-b18c27449109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.819541 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c5bf2-ef58-4f55-bf46-b18c27449109 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.845722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1446/2886] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Jul 30 05:10:36 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 30 05:10:36.849804 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7795ac-2a04-4905-98f8-3239fcb6cf66 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.860944 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7795ac-2a04-4905-98f8-3239fcb6cf66 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.917892 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a58f-567b-47c8-a7aa-957d3ff8dba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.918533 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a58f-567b-47c8-a7aa-957d3ff8dba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.918818 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a58f-567b-47c8-a7aa-957d3ff8dba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.919095 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a58f-567b-47c8-a7aa-957d3ff8dba5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.919766 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6a58f-567b-47c8-a7aa-957d3ff8dba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.921689 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6a58f-567b-47c8-a7aa-957d3ff8dba5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.943064 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1441/2887] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Jul 30 05:10:36 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 30 05:10:36.947187 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c5bf2-ef58-4f55-bf46-b18c27449109 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.958169 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c5bf2-ef58-4f55-bf46-b18c27449109 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.962901 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ddd33-d389-4fc7-8b2a-b542a5b97888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.963504 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ddd33-d389-4fc7-8b2a-b542a5b97888 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.963784 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ddd33-d389-4fc7-8b2a-b542a5b97888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.964057 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ddd33-d389-4fc7-8b2a-b542a5b97888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:36.964749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ddd33-d389-4fc7-8b2a-b542a5b97888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:36.966355 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ddd33-d389-4fc7-8b2a-b542a5b97888 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:36.977036 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1447/2888] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:36.981061 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a58f-567b-47c8-a7aa-957d3ff8dba5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:36.991152 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a58f-567b-47c8-a7aa-957d3ff8dba5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:36.995789 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0eed0-8643-4ba6-9cab-d957a3f17e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:36.996441 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0eed0-8643-4ba6-9cab-d957a3f17e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:36.996725 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0eed0-8643-4ba6-9cab-d957a3f17e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:36.997005 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0eed0-8643-4ba6-9cab-d957a3f17e65 admin admin] PATH_INFO: `/v3/projects/75de9e7194654ef893dfcc4af055965d/users/07741292a6114ec086c17c05cec8ae0d/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:37.010231 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:37.013671 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0eed0-8643-4ba6-9cab-d957a3f17e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75de9e7194654ef893dfcc4af055965d, user_id=07741292a6114ec086c17c05cec8ae0d, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:37.017849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0eed0-8643-4ba6-9cab-d957a3f17e65 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:37.031407 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1442/2889] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:36 2022] PUT /identity/v3/projects/75de9e7194654ef893dfcc4af055965d/users/07741292a6114ec086c17c05cec8ae0d/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:37.035413 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ddd33-d389-4fc7-8b2a-b542a5b97888 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:37.045796 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ddd33-d389-4fc7-8b2a-b542a5b97888 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:37.087164 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:37.102747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0d396-8c1f-45d5-b85f-2deb2c32d866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:37.103360 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0d396-8c1f-45d5-b85f-2deb2c32d866 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:37.103642 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0d396-8c1f-45d5-b85f-2deb2c32d866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:37.103914 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0d396-8c1f-45d5-b85f-2deb2c32d866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:37.104604 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0d396-8c1f-45d5-b85f-2deb2c32d866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:37.106199 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0d396-8c1f-45d5-b85f-2deb2c32d866 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:37.116258 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1448/2890] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Jul 30 05:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 30 05:10:37.120431 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0eed0-8643-4ba6-9cab-d957a3f17e65 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:37.131246 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0eed0-8643-4ba6-9cab-d957a3f17e65 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:37.135888 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298d73b-5d22-4713-b3fe-74035d34896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:37.136535 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298d73b-5d22-4713-b3fe-74035d34896d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:37.136811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298d73b-5d22-4713-b3fe-74035d34896d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:37.137078 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298d73b-5d22-4713-b3fe-74035d34896d admin admin] PATH_INFO: `/v3/projects/75de9e7194654ef893dfcc4af055965d/users/07741292a6114ec086c17c05cec8ae0d/roles/c09e805a3a134c229ec9d02fe9bea8cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:37.139043 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298d73b-5d22-4713-b3fe-74035d34896d admin admin] RBAC: Authorizing `identity:create_grant(project_id=75de9e7194654ef893dfcc4af055965d, user_id=07741292a6114ec086c17c05cec8ae0d, role_id=c09e805a3a134c229ec9d02fe9bea8cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:37.141836 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298d73b-5d22-4713-b3fe-74035d34896d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:37.151869 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1443/2891] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Jul 30 05:10:37 2022] PUT /identity/v3/projects/75de9e7194654ef893dfcc4af055965d/users/07741292a6114ec086c17c05cec8ae0d/roles/c09e805a3a134c229ec9d02fe9bea8cb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:37.162290 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4c4c0-3962-4d75-be34-45d3b58464de None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:37.162692 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4c4c0-3962-4d75-be34-45d3b58464de None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:37.163244 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4c4c0-3962-4d75-be34-45d3b58464de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:37.191006 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-86d4c4c0-3962-4d75-be34-45d3b58464de None None] MFA Rules not processed for user `07741292a6114ec086c17c05cec8ae0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:37.192207 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-86d4c4c0-3962-4d75-be34-45d3b58464de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:37.258794 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1449/2892] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:37.270207 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298d73b-5d22-4713-b3fe-74035d34896d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:37.271990 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0298d73b-5d22-4713-b3fe-74035d34896d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:37.329702 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298d73b-5d22-4713-b3fe-74035d34896d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:37.333367 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec0370-31d8-4944-b5f4-6a8fb7026098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:37.333975 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec0370-31d8-4944-b5f4-6a8fb7026098 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:37.334251 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec0370-31d8-4944-b5f4-6a8fb7026098 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:37.334574 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec0370-31d8-4944-b5f4-6a8fb7026098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:37.391060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec0370-31d8-4944-b5f4-6a8fb7026098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:37.392384 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec0370-31d8-4944-b5f4-6a8fb7026098 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:37.401203 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1444/2893] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:38.902905 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4c4c0-3962-4d75-be34-45d3b58464de None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:38.904317 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-86d4c4c0-3962-4d75-be34-45d3b58464de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:38.916444 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4c4c0-3962-4d75-be34-45d3b58464de None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:38.919962 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918442f8-9bab-4ff0-b70f-66c8acdf5fca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:38.920558 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918442f8-9bab-4ff0-b70f-66c8acdf5fca service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:38.920832 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918442f8-9bab-4ff0-b70f-66c8acdf5fca service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:38.921101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918442f8-9bab-4ff0-b70f-66c8acdf5fca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:38.978204 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918442f8-9bab-4ff0-b70f-66c8acdf5fca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:38.979629 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918442f8-9bab-4ff0-b70f-66c8acdf5fca service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:38.989015 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1450/2894] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:10:39.521483 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec0370-31d8-4944-b5f4-6a8fb7026098 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:39.532638 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec0370-31d8-4944-b5f4-6a8fb7026098 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:39.536657 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdd794-81b6-4501-8e67-59ba191105e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0d2552bd0e244dea55eb1eecf471fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:39.537277 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdd794-81b6-4501-8e67-59ba191105e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:39.537588 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdd794-81b6-4501-8e67-59ba191105e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:39.537877 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdd794-81b6-4501-8e67-59ba191105e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:39.551264 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fdd794-81b6-4501-8e67-59ba191105e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:39.552425 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fdd794-81b6-4501-8e67-59ba191105e3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:39.563885 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1445/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 30 05:10:43.433492 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918442f8-9bab-4ff0-b70f-66c8acdf5fca service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:43.494364 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918442f8-9bab-4ff0-b70f-66c8acdf5fca service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:43.498846 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:43.499466 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:43.499741 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:43.500570 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] PATH_INFO: `/v3/users/3fee48aadc65451a903ef132bc5c14ff` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:43.501749 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fee48aadc65451a903ef132bc5c14ff)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:43.503670 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:43.508119 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:43.549747 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fee48aadc65451a903ef132bc5c14ff'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:43.556840 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fee48aadc65451a903ef132bc5c14ff'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:43.557501 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fee48aadc65451a903ef132bc5c14ff'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:43.558115 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fee48aadc65451a903ef132bc5c14ff'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:43.570475 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fee48aadc65451a903ef132bc5c14ff'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:43.583735 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1451/2896] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:43 2022] DELETE /identity/v3/users/3fee48aadc65451a903ef132bc5c14ff => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:43.588216 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eddb65-165e-4da4-8111-33ba202cbc91 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:43.588674 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eddb65-165e-4da4-8111-33ba202cbc91 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:43.588872 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eddb65-165e-4da4-8111-33ba202cbc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:43.607652 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-26eddb65-165e-4da4-8111-33ba202cbc91 None None] MFA Rules not processed for user `74c09f97e34c428a84e13d34f5105ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 30 05:10:43.608866 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-26eddb65-165e-4da4-8111-33ba202cbc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:43.667829 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1446/2897] 10.222.0.20 () {58 vars in 958 bytes} [Sat Jul 30 05:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 30 05:10:43.677527 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:43.678955 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:43.735410 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85f525-a006-447f-90f0-657d441e0e70 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:43.738979 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11bff3-22db-4bbb-9c4f-e1348e411e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426be56424504e958402d785a068d2f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:43.739581 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11bff3-22db-4bbb-9c4f-e1348e411e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:43.739855 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11bff3-22db-4bbb-9c4f-e1348e411e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:43.740121 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11bff3-22db-4bbb-9c4f-e1348e411e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:43.749546 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11bff3-22db-4bbb-9c4f-e1348e411e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:43.751542 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11bff3-22db-4bbb-9c4f-e1348e411e2e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:43.761143 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1452/2898] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:10:44.030455 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eddb65-165e-4da4-8111-33ba202cbc91 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:44.032112 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-26eddb65-165e-4da4-8111-33ba202cbc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:44.046470 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eddb65-165e-4da4-8111-33ba202cbc91 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:44.052823 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:44.053736 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:44.054019 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:44.054295 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] PATH_INFO: `/v3/projects/8ec7e37cf7e845c79534c5f8955af5ee` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:44.055755 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec7e37cf7e845c79534c5f8955af5ee)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:44.058123 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:44.075849 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:44.106988 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec7e37cf7e845c79534c5f8955af5ee', 'request_id': 'req-2f7793e3-9950-45d5-8a89-a292f187fc76'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.114515 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec7e37cf7e845c79534c5f8955af5ee', 'request_id': 'req-2f7793e3-9950-45d5-8a89-a292f187fc76'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.120140 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec7e37cf7e845c79534c5f8955af5ee', 'request_id': 'req-2f7793e3-9950-45d5-8a89-a292f187fc76'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.131526 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] The token cache is being invalidate because project 8ec7e37cf7e845c79534c5f8955af5ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:44.131811 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.132459 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.133947 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1447/2899] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:44 2022] DELETE /identity/v3/projects/8ec7e37cf7e845c79534c5f8955af5ee => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:44.137935 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11bff3-22db-4bbb-9c4f-e1348e411e2e service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:44.147767 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11bff3-22db-4bbb-9c4f-e1348e411e2e service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:44.204652 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:44.205188 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:44.205470 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:44.205745 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] PATH_INFO: `/v3/users/e99a817656544ccb8a696bf7340b210b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:44.206933 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99a817656544ccb8a696bf7340b210b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:44.208849 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:44.236852 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:44.255763 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99a817656544ccb8a696bf7340b210b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.262835 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99a817656544ccb8a696bf7340b210b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.263507 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99a817656544ccb8a696bf7340b210b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.264101 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99a817656544ccb8a696bf7340b210b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.275714 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99a817656544ccb8a696bf7340b210b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.289165 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1453/2900] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:44 2022] DELETE /identity/v3/users/e99a817656544ccb8a696bf7340b210b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:44.556759 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:44.558422 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:44.573619 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7793e3-9950-45d5-8a89-a292f187fc76 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:44.632204 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:44.632851 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:44.633120 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:44.633397 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] PATH_INFO: `/v3/projects/88ef764f43644c279abd7b1b8f4a0a47` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:44.634597 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88ef764f43644c279abd7b1b8f4a0a47)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:44.636626 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:44.683957 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88ef764f43644c279abd7b1b8f4a0a47', 'request_id': 'req-fa127db8-ba9f-40fe-a05b-b188850bccd8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.691103 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88ef764f43644c279abd7b1b8f4a0a47', 'request_id': 'req-fa127db8-ba9f-40fe-a05b-b188850bccd8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.696628 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88ef764f43644c279abd7b1b8f4a0a47', 'request_id': 'req-fa127db8-ba9f-40fe-a05b-b188850bccd8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.708060 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] The token cache is being invalidate because project 88ef764f43644c279abd7b1b8f4a0a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:44.708355 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.708982 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.710486 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1448/2901] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:44 2022] DELETE /identity/v3/projects/88ef764f43644c279abd7b1b8f4a0a47 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:44.714431 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:44.715974 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:44.729792 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25aeec-3c3c-456e-98d0-81ed3d973234 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:44.787675 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:44.788565 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:44.788843 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:44.789114 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] PATH_INFO: `/v3/users/07741292a6114ec086c17c05cec8ae0d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:44.790271 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] RBAC: Authorizing `identity:delete_user(user_id=07741292a6114ec086c17c05cec8ae0d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:44.792240 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:44.837173 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07741292a6114ec086c17c05cec8ae0d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.844651 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07741292a6114ec086c17c05cec8ae0d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.845478 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07741292a6114ec086c17c05cec8ae0d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.846079 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07741292a6114ec086c17c05cec8ae0d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.857870 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07741292a6114ec086c17c05cec8ae0d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:44.870306 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1454/2902] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Jul 30 05:10:44 2022] DELETE /identity/v3/users/07741292a6114ec086c17c05cec8ae0d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:45.150067 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:10:45.151639 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:10:45.165348 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa127db8-ba9f-40fe-a05b-b188850bccd8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:10:45.197991 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:45.227373 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c09f97e34c428a84e13d34f5105ca9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e75ff1f24d24ead87dfc7d32ddc1723', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:10:45.228029 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:10:45.228314 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:10:45.228622 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] PATH_INFO: `/v3/projects/75de9e7194654ef893dfcc4af055965d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:10:45.229787 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75de9e7194654ef893dfcc4af055965d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:10:45.231732 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:10:45.279009 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75de9e7194654ef893dfcc4af055965d', 'request_id': 'req-78249d07-767b-44c0-a865-8b5b9aadff67'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:45.286749 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75de9e7194654ef893dfcc4af055965d', 'request_id': 'req-78249d07-767b-44c0-a865-8b5b9aadff67'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:45.292542 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75de9e7194654ef893dfcc4af055965d', 'request_id': 'req-78249d07-767b-44c0-a865-8b5b9aadff67'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:45.303292 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] The token cache is being invalidate because project 75de9e7194654ef893dfcc4af055965d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 30 05:10:45.303577 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:45.304193 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78249d07-767b-44c0-a865-8b5b9aadff67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 30 05:10:45.305678 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1449/2903] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Jul 30 05:10:45 2022] DELETE /identity/v3/projects/75de9e7194654ef893dfcc4af055965d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 30 05:10:54.866864 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=72,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:10:55.300420 dv-cinder-851279-3-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 30 05:12:28.294722 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 30 05:12:28.296174 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:12:28.364706 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca2f1f-4ebf-4ecf-b418-4e74d644936f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 30 05:12:28.423418 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467164b-5f7d-49be-bac8-c59d43a05bc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21b7c64636bd47e6b9a486b6cdaab4b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2bfc8c66b48b4981832f2c4b940df124', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 30 05:12:28.424043 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467164b-5f7d-49be-bac8-c59d43a05bc3 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 30 05:12:28.424324 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467164b-5f7d-49be-bac8-c59d43a05bc3 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 30 05:12:28.424598 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467164b-5f7d-49be-bac8-c59d43a05bc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 30 05:12:28.426353 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c467164b-5f7d-49be-bac8-c59d43a05bc3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 30 05:12:28.438248 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467164b-5f7d-49be-bac8-c59d43a05bc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 30 05:12:28.439857 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467164b-5f7d-49be-bac8-c59d43a05bc3 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 30 05:12:28.449694 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1455/2904] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Jul 30 05:12:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 30 05:12:38.445451 dv-cinder-851279-3-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=58 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}